From: maj@waikato.ac.nz
Newsgroups: bit.listserv.stat-l
Subject: Re: F tests when scores are responses
Date: 10 Oct 92 16:54:04 +1300
Organization: University of Waikato, Hamilton, New Zealand

In article <1992Oct9.132428.11334@waikato.ac.nz>, maj@waikato.ac.nz writes:
> Dear Stat-L,
> 
> I have a data set taking the form of a large, unbalanced ANOVA
> situation. 
> 
>     It's large : around 5000 observations
> 
> 
>     There are five factors having 19,5,5,18,4 levels
> 
> 
> I want to analyse this using regression programs by using
> groups of indicator variables for the factors, in fact this
> has already been done. There may be interactions but I have
> restricted attention to two-factor interactions not involving
> the many-levelled factors.
> 
> The problem is this: the response variable is not continuous,
> but is a subjective score on a 3-point scale.
> 
> 
> My request is: does anyone have references to the behaviour of
> General Linear tests with this type of data?
> 
> [Cochran,W.G. (1950) 'The comparison of percentages in
> matched samples' Biometrika 37,256-266
> 
> did some simulations with binary responses which seemed to
> suggest that the usual F critical values were not too misleading,
> but there must be some more recent references.]
I had quite a few responses, but most were trying to get me
to do things 'properly' via logistic or polytomous regression.
I was interested to learn that SAS had facilities for
logistic regression with ordinal scores.

This is well-taken, but the data set is very large and messy
and it seem sensible and practical to do at least the first
phase of the model-selection in the linear model framework,
before finishing up with more appropriate but less routine
procedures.

I'm not the only person to do this, Allen Herzog writes:

          When I have large numbers of dichotomous variables in a one-way
          anova set-up (and occasionally in more complex cases) and if the
          the proportions of the dichotomous variables for each level of
          the independent variables do not lie above .15 or below .85 then
          I use Anova as a preliminary test before turning to the more
          expensive logistic regression.


Some wondered about the response scale: it is a rating into
3 categories [Good,Marginal,Unacceptable]. I believe that it
is reasonable to analyse 1,2,3 codes as if they were a measured
variable. Actually this hints at another complication: not
all 5000 odd observations were by the same rater and the raters
are really random effects. Because of client confidentiality I
don't wish to discuss the data in any detail.

Another complication from my point of view is that this is a
student project that I am supervising, which tends to make any
computing difficulties an order of magnitude more troublesome 
to cope with.

My thanks to those who responded:

    Stephanie Watts Butler     butler@spdc.ti.com
    Texas Instruments

    DMR@PSUVM.PSU.EDU     Dennis Roberts

    Meredith_Warshaw@frankston.com

    wiorkow@utdallas.edu      John J. Wiorkowski

    FDANE@uga.cc.uga.edu       Frank Dane

    Philip Dixon, dixon@srel.edu

I'm going to go on being 'quick and dirty' with this dataset for
a while, at the moment I'm getting my student to produce Mallows'
Cp plots for a large number of (linear) models for the data. I'd
value any references that could help me judge whether my sins are
mortal or venial.
-- 
Murray A. Jorgensen [ maj@waikato.ac.nz ]    University of Waikato
Department of Mathematics and Statistics     Hamilton, New Zealand      
__________________________________________________________________
'Tis the song of the Jubjub! the proof is complete,
if only I've stated it thrice.'
Comments: Gated by NETNEWS@AUVM.AMERICAN.EDU