Newsgroups: bit.listserv.stat-l
Date:         Sat, 11 Apr 1992 13:47:00 +0930
From:         Bill Venables <wvenable@STATS.ADELAIDE.EDU.AU>
Subject:      logistic regression with ungrouped data
X-To:         STAT-L@VM1.MCGILL.CA
X-cc:         mcphers@HILBERT.MATHS.UTAS.EDU.AU
              11 Apr 1992 11:10:46 EST
              <9204110117.AA01876@spam.maths.adelaide.edu.au>

Glen McPherson asks

>
> 	     LOGISTIC REGRESSION WITH UNGROUPED DATA
>
>
> The Likelihood ratio statistic (Deviance, G squared, etc.) has a distribution
> which is well approximated by a Chi-squared distribution when based on the
> comparison of two models. However, it has been reported (e.g. McCullagh &
> Nelder's book on Generalized linear models) that the Chi-squared approximation
> is not adequate for the comparison of models with ungrouped data even when
> sample sizes are large. (In the illustration quoted above, for example, the
> residual degrees of freedom are 2,480.)
>
> If there is a single explanatory (regressor) variable, it is feasible to
> order and group the values of this variable and transform the problem to
> one in which there is grouped data. Then, there is the possibility of using
> both the Goodness of fit statistic and residual analysis to judge the
> adequacy of fit of a model to the data. When there are more than 2 or 3
> variables in the regression equation, this is not a feasible option because
> the number of groups to be formed is too great. Can anyone provide a method
> which can be applied for ungrouped data, or suggest a possible path which
> we might investigate?

Likelihood Ratio tests are well approximated by a Chi-squared test when
they correspond to a test of one hypothesis within another, both of which
have a fixed number of parameters, say p0 and p, as the sample size, say n,
(or sample information for non-iid cases) increases.  Moreover the
approximation only starts to become reasonable if p << n.  The deviance is
a likelihood ratio test of a given model within a saturated model, that is,
the model with one parameter for each observation.

In the case of grouped data, the number of `observations' is the number of
groups, p, which remains fixed as the sample size increases, and hence the
asymptotic argument applies.  In the ungrouped case, each new response
brings a new parameter to the saturated model, and so here p = n, the
argument does not apply, and the Chi-squared approximation is often very
poor.  In my experience what happens in logistic regression is usually that
the deviance is {\em well below\/} expectation, giving (the naive) the
impression of a superexcellent fit.

Nevertheless it seems to me that the deviance is a natural test statistic
to use for testing the fit, but its null distribution is unknown.  Given
the model, the MLE is likely to be quite OK.  So as a stab at the problem
in a fixed case why not simulate the distribution of the deviance, using
the MLEs as parameters, and checking where the observed deviance sits with
respect to its simulated null distribution?  This simply amounts to
selecting new binary samples with known probabilities and re-calculating
the MLE and deviance for each, which may be computationally intensive, of
course, but not an insurmountable problem.

Certain refinements are possible.  For example you might reject simulated
binary samples that give an MLE differing too much from the one you
started with, since really what you would like to do is condition on the
minimal sufficient statistic, and the MLE is presumably a good surrogate.

Let me be clear - I offer this as a speculative suggestion for
investigation only.  They're may be hidden logic traps that I have
overlooked, but from my practical experience it does seem to be OK.

--
Bill Venables, Dept. of Statistics, | Email: venables@stats.adelaide.edu.au
Univ. of Adelaide, South Australia. | Tel: +61 8 228 5412  Fax: ...232 5670
Comments: Gated by NETNEWS@AUVM.AMERICAN.EDU