Newsgroups: bit.listserv.spssx-l
Subject: Re: GLM's SS3 ala Dallal
From: nichols@spss.com (David Nichols)
Date: Wed, 9 Sep 1992 23:27:32 GMT
Organization: SPSS Inc.

My apologies for not getting this out sooner, but we're in the middle of
a new release and I've been swamped. Anyway, here is what's going on in
MANOVA in Jerry Dallal's _American Statistician_ letter:

The design is a 2 x 3 with six subjects in each of the three cells in row
one and three subjects in each of the three cells in row two. Jerry Dallal
ran this as a straight A by B design in SAS' PROC GLM and SPSS' MANOVA,
and of course got the same answers for SAS' Type III and MANOVA's UNIQUE
sums of squares. He then included a third factor C, nested under A by B,
as a way of explicitly fitting the within cells error term as a factor.
When he did this, the results in SPSS and SAS were different than before
and different than each other. The question is, what is going on here?

I can't speak for SAS and am not that familiar with their generalized
inverse algorithm, so I don't know what they do in this situation. However,
for MANOVA what happened was that the commands must have been run essentially
like the following in order to get the output in the letter:

manova y by a(1,2) b(1,3) c(1,6)
 /des=a, b, a by b, c w a by b

The problem here is that the C factor does not have 6 levels at each
combination of A by B. This results in the equivalent of an unbalanced
nested design. More specifically for MANOVA, it results in redundancies
in the design matrix, which wreak havoc with UNIQUE sums of squares, and
produce essentially useless results. The following warning would have
been produced by the job:

 Redundancies in Design Matrix
    Column     Effect

         9    C W A BY B
        10     (SAME)
        11     (SAME)
        14     (SAME)
        15     (SAME)
        16     (SAME)
        19     (SAME)
        20     (SAME)
        21     (SAME)

 - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
 * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *
 *                   *                                                 *
 *   W A R N I N G   * UNIQUE sums-of-squares are obtained assuming    *
 *                   * the redundant effects (possibly caused by       *
 *                   * missing cells) are actually null.               *
 *                   * The hypotheses tested may not be the            *
 *                   * hypotheses of interest.  Different reorderings  *
 *                   * of the model or data, or different contrasts    *
 *                   * may result in different UNIQUE sums-of-squares. *
 *                   *                                                 *
 * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *
 - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

Any time you see this warning in MANOVA, you have a problem. You should
never accept results at face value if you see this warning. It is really
telling you that (in almost every case) what you are getting is useless.

If the design had had the same number of cases in each cell, C would have
had the same number of levels within each A by B nesting, and there would
have been no problem. There is a way to deal with this situation, though
it is very tedious for large designs. It is the same method used to deal
with any unbalanced nested design or factorial design with empty cells
(both of which create design redundancies): lay the data out as a one way
design, fit SPECIAL contrasts among appropriate cells of the one way
design, partition the term into appropriate separate terms, and list each
one on the DESIGN subcommand. Here's how we can make MANOVA give us the
desired results:

data list free / obs a b c y
begin data
  1    1    1     1    3.81
  2    1    1     2    4.64
  3    1    1     3    4.09
  4    1    2     4    3.42
  5    1    2     5    3.57
  6    1    2     6    3.55
  7    1    3     7    3.55
  8    1    3     8    3.71
  9    1    3     9    3.66
 10    2    1    10    0.22
 11    2    1    11    0.33
 12    2    1    12    0.36
 13    2    1    13    1.08
 14    2    1    14    1.33
 15    2    1    15    1.15
 16    2    2    16    0.36
 17    2    2    17    0.27
 18    2    2    18    0.26
 19    2    2    19    0.83
 20    2    2    20    0.90
 21    2    2    21    0.93
 22    2    3    22    0.37
 23    2    3    23    0.31
 24    2    3    24    0.28
 25    2    3    25    0.70
 26    2    3    26    0.89
 27    2    3    27    0.93
end data
set width=80

manova y by c(1,27)
 /contrast(c)=special(27*1,
  2 2 2 2 2 2 2 2 2 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1
  2 2 2 0 0 0 -2 -2 -2 1 1 1 1 1 1 0 0 0 0 0 0 -1 -1 -1 -1 -1 -1
  0 0 0 2 2 2 -2 -2 -2 0 0 0 0 0 0 1 1 1 1 1 1 -1 -1 -1 -1 -1 -1
  2 2 2 0 0 0 -2 -2 -2 -1 -1 -1 -1 -1 -1 0 0 0 0 0 0 1 1 1 1 1 1
  0 0 0 2 2 2 -2 -2 -2 0 0 0 0 0 0 -1 -1 -1 -1 -1 -1 1 1 1 1 1 1
  1 0 -1 24*0
  0 1 -1 24*0
  3*0 1 0 -1 21*0
  3*0 0 1 -1 21*0
  6*0 1 0 -1 18*0
  6*0 0 1 -1 18*0
  9*0 1 0 0 0 0 -1 12*0
  9*0 0 1 0 0 0 -1 12*0
  9*0 0 0 1 0 0 -1 12*0
  9*0 0 0 0 1 0 -1 12*0
  9*0 0 0 0 0 1 -1 12*0
  15*0 1 0 0 0 0 -1 6*0
  15*0 0 1 0 0 0 -1 6*0
  15*0 0 0 1 0 0 -1 6*0
  15*0 0 0 0 1 0 -1 6*0
  15*0 0 0 0 0 1 -1 6*0
  21*0 1 0 0 0 0 -1
  21*0 0 1 0 0 0 -1
  21*0 0 0 1 0 0 -1
  21*0 0 0 0 1 0 -1
  21*0 0 0 0 0 1 -1)
 /partition(c)=(1,2,2,21)
 /design=c(1) c(2) c(3) c(4)

Now we get the results we want in the ANOVA table:

 - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
 * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *
 *                   *                                                 *
 *   W A R N I N G   *  Too few degrees of freedom in RESIDUAL         *
 *                   *  error term (DF = 0).                           *
 *                   *                                                 *
 * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *

 - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

* * * * * * A N A L Y S I S   O F   V A R I A N C E -- DESIGN   1 * * * * * *

 Tests of Significance for Y using UNIQUE sums of squares
 Source of Variation          SS      DF        MS         F  Sig of F

 RESIDUAL                    .00       0       .
 C(1)                      59.12       1     59.12       .        .
 C(2)                        .79       2       .39       .        .
 C(3)                        .28       2       .14       .        .
 C(4)                       2.57      21       .12       .        .
 - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

--
 David Nichols           Statistical Support Specialist            SPSS, Inc.
 Phone: (312) 329-3684     Internet: nichols@spss.com     Fax: (312) 329-3657
*******************************************************************************
 Any correlation between my views and those of SPSS is strictly due to chance.
Comments: Gated by NETNEWS@AUVM.AMERICAN.EDU