From: nichols@spss.com (David Nichols)
Subject: SPSS with unequal N
Date: Fri, 17 Apr 1992 15:37:17 GMT
Organization: SPSS Inc.

Some recent bitnet discussions concerning unequal cell sizes in analysis
of variance (anova), combined with the fact that we here at SPSS get a
large number of technical support calls dealing with this topic, have
prompted me to put together a (hopefully reasonably succinct) summary
of current and past states of affairs with regard to how SPSS procedures
handle this situation. 

In particular, in the context of comparisons between SPSS and SAS that
took place recently on the sas-l list, it was stated that at one time
SPSS had an anova procedure which assumed equal cell sizes, and that a
whole generation of social scientists had been publishing incorrect
results because of this. This is totally false. I contacted the poster
by e-mail and he admitted that he did not know what procedure this was.
After doing some checking, I'm able to confirm that SPSS has never had
any such procedure. SAS' PROC ANOVA does indeed assume equal cell sizes
and will produce potentially useless results when fed unbalanced data.
However, this is quite clearly stated in the manual, along with the
proper admonition that the procedure is designed solely for balanced
designs and the user is responsible for the results if he or she chooses
to give it unbalanced data.

As for handling of anova in SPSS: There are three primary procedures in
SPSS designed for anova: ONEWAY, ANOVA and MANOVA. There are also other
procedures such as MEANS, RELIABILITY and QUICK CLUSTER where specific
types of anova are performed. The only case in which balance is required
is in the simple one way repeated measures anova done by RELIABILITY.
Provided that the user enters actual data (as opposed to setting up a
bogus matrix to enter) the listwise deletion used in RELIABILITY makes it
impossible to get incorrect results. The analyses in QUICK CLUSTER are
simply one way anovas on each clustering variable; they do not assume
equal cluster sizes, but should only be used for descriptive purposes
anyway, since they are done after creating clusters rather than as a
test of a null hypothesis about group means.

The MEANS and ONEWAY procedures both produce one way anovas based on
the standard formula used to test the null hypothesis that all means
are equal. With regard to the omnibus F-test for a one way design, the
normally used different types of sums of squares all collapse to the
same thing. Results from MEANS or ONEWAY (or ANOVA or MANOVA) for the
one way case should match each other and pretty much everyone else's
software as well. Note that there is a method discussed in some texts
(e.g., Winer, Keppel) called the "method of unweighted means." This
method is purported to be useful when differences in cell sizes are not
related to population proportions (the usual case, I think). I'm not
entirely sure of the history of this method, but it is an approximate
analysis from pre-computer times. I have not yet found a software
package that does this analysis.

When factorial designs are involved, the situation becomes more complex.
Different algorithms exist for fitting different types of sums of squares.
The approach generally recommended by most statisticians as long as there
are no empty cells is what SPSS calls unique or regression approach sums of
squares. These sums of squares adjust each term in the model for every other
term. They do not add up to the total sum of squares. They are generally
preferred because they test the same hypotheses about unweighted cell means
that are tested with balanced data. This type of analysis is also known as
Yates' weighted squares of means analysis. SAS refers to them as Type III
sums of squares.

Unique sums of squares are the default in the MANOVA procedure for versions
2.2 and later of SPSS-X (beginning with 4.0 it is no longer SPSS-X, just
SPSS), and on all micro platforms. The other option in MANOVA is the
sequential decomposition approach. Some refer to this as a hierarchical
approach (note though that the HIERARCHICAL option in ANOVA is not always
the same as SEQUENTIAL in MANOVA). SAS calls these Type I sums of squares.
MANOVA also has a balanced option, specified by METHOD=ESTIMATION(BALANCED).
This will cause MANOVA to use a simpler, faster algorithm for analyzing
balanced data. However, even if this is specified, MANOVA will check the
data and will revert to its standard algorithm if the data are unbalanced.

ANOVA offers a whole host of options for handling unbalanced data. My
recommendation is that if all cells have at least one observation, use the
unique option unless you have a very good reason for choosing something else.
The current default is referred to as the classic experimental approach, with
covariates entered prior to factors, main effects simultaneously, and each
order of interactions after lower order terms but simultanesouly within a
level (that is, covariates first, then main effects together, then two way
interactions together, etc.). The reasoning behind this method is that one
should first test higher order interactions (note that all methods give the
same results for the last term in the model). If these are significant, then
standard tests on the lower order terms are not carried out because these
effects are confounded with higher order terms. If these are not significant,
then one precedes to the next lower order assuming the higher order null,
and tests terms adjusted only for equal and lower orders. Though this method
has a logic to it, I don't recommend it, because if higher order terms are
really null, one should remove them and reestimate the model, which will
result in a different error term. In two way and some other simple designs,
these can match SAS Type II sums of squares. Given that this method seldom
tests the hypotheses of interest to most users, we are changing the default
in version 5 to unique on all platforms. The current default results are
not "wrong" and do not assume equal cell sizes; they simply (correctly) test
hypotheses that are seldom of interest.

The empty cells situation (which can also be created for computational
purposes by an unbalanced nested design) is the most difficult situation
to deal with. This only occurs with more than one factor, since categories
missing from a one way design simply create one less level in the factor.
There are no adequate provisions for this case in ANOVA. MANOVA can be
made to produce appropriate results with a little work (as with SAS' Type
IV sums of squares). Users should thoroughly inspect their data to see
that they have at least one observation per cell. If you have missing cells
and aren't VERY proficient in statistics, consult someone who is. Though
neither of them is particularly useful as a guide for using SPSS, there
are two books that I know of that are helpful in understanding the general
situation vis a vis unbalanced data. They are:

_Analysis of Messy Data: Volume 1: Designed Experiments_ by George A.
  Milliken and Dallas E. Johnson. 1984: Van Nostrand Reinhold Company.

_Linear Models for Unbalanced Data_ by S. R. Searle. 1987: Wiley.

Searle doesn't even discuss MANOVA, I don't think. Milliken and Johnson 
do somewhat, but they mistakenly state (p. 190) that it will not produce
Type IV sums of squares. I can show interested users how to do this.

For SPSS users who have further questions, particularly concerning
analyzing data with empty cells, you can contact me.
--
 David Nichols           Statistical Support Specialist            SPSS, Inc.
 Phone: (312) 329-3684     Internet: nichols@spss.com     Fax: (312) 329-3657
*******************************************************************************
 Any correlation between my views and those of SPSS is strictly due to chance.
Comments: Gated by NETNEWS@AUVM.AMERICAN.EDU