Newsgroups: bit.listserv.spssx-l
Subject: Re: COMPUTATIONAL ERROR IN SPSS ONEWAY
From: nichols@spss.com (David Nichols)
Date: Tue, 25 Aug 1992 17:22:31 GMT
Organization: SPSS Inc.


Can anyone explain the following which I received from one of our users?

----------------------------Original message----------------------------

At RIA we are using SPSS RELEASE 4.1 FOR VAX/VMS (VMS V5.3).  There are
actually two types of errors when using the ONEWAY with RANGES option.
First, the program will report significant pairwise comparisons when the
overall equation approaches but fails to reach significance (i.e., p = .053).
Second, it continues to test intermediate pairs even when the largest/smallest
pairs are not significantly different.  Below is an example this pattern:

   (*) DENOTES PAIRS OF GROUPS SIGNIFICANTLY DIFFERENT AT THE 0.050 LEVEL

                     G  G  G
                     r  r  r
                     p  p  p

   Mean     Group    2  1  3

  28.5934   Grp 2
  32.1382   Grp 1    *
  33.5813   Grp 3

According to Winer (1971), if the two most extreme pairs are not significant,
no further testing should be done for any intermediate scores between those
two extreme scores.

-----------------------------------------------------------------------------

There are two issues in the user message posted by Necia Black. One
of these is an error by SPSS under some circumstances; the other is
user misunderstanding.

There are seven different types of tests offered in ONEWAY under the
RANGES subcommand. Three of these (Duncan, Student-Newman-Keuls and
TukeyB) are sequential tests, in which intermediate means should not
be tested if a larger set of nonsignificantly different means contains
them. These tests are known to be valid in this manner only for equal
sample sizes (you will probably see a good deal of disagreement about
what to do when sample sizes are unequal). 

ONEWAY potentially gives two pieces of output for RANGES: the pairwise
table shown in the posting, and homogeneous subsets output, showing
sets of means that do not differ significantly from one another. The
homogeneous subsets output only occurs when sample sizes are equal
or the HARMONIC=ALL subcommand is used (effectively making sample
sizes equal). This output is always consonant with recommendations
made by the developers of these tests and sources like Winer.

There is an error that can occur in versions previous to 5.0 (now
available only on Windows) in the pairwise output, though it is
fairly rare. Because of the sequential nature of the three tests
mentioned above, all pairwise comparisons should not be made unless
containing comparisons are significant. Versions of SPSS prior to
5.0 are making all pairwise comparisons. However, in the vast majority
of cases the pairwise output matches the (correct) homogeneous subsets
output. 

So, to summarize: this question is only relevant with equal sample
sizes or using the harmonic mean and employing one of the three tests
that are sequential in nature (Duncan, SNK, TukeyB). It is also very
rare even under these circumstances for an error to occur. The other
four tests (LSD, MODLSD=Bonferroni, TUKEY and SCHEFFE) are not 
sequential, so this issue does not occur. 

There will likely be those who will criticize us for choosing to
conduct all pairwise comparisons for the sequential tests with unequal
sample sizes, as we are continuing to do. However, since the original
logic under which the tests were derived no longer holds, we have
chosen to provide users with all the information and allow them to
use it as they see fit; homogeneous subsets output can always be
created from the pairwise table.

The other question involves an error by the user. That is, there is no
problem involved with the fact that ONEWAY will report significant
pairwise comparisons when the overall F is nonsignificant (unless it
were to do so for Scheffe comparisons, which it will not). There is
only one necessary relationship between the overall F and pairwise
comparsions: if the overall F is nonsignificant, there will be no
significant Scheffe comparisons (pairwise or other). Aside from this,
nothing else is guaranteed.

--
 David Nichols           Statistical Support Specialist            SPSS, Inc.
 Phone: (312) 329-3684     Internet: nichols@spss.com     Fax: (312) 329-3657
*******************************************************************************
 Any correlation between my views and those of SPSS is strictly due to chance.
Comments: Gated by NETNEWS@AUVM.AMERICAN.EDU