From: s-sender@utstat.toronto.edu
Subject: news from S-news
X-Gateway: dmssyd.syd.dms.CSIRO.AU
Organization: CSIRO Div. Maths & Stats, CSIRO Div. Applied Physics
Date: Thu, 13 Aug 1992 00:08:33 GMT

Received: by utstat; Wed Aug 12 18:23 EDT 1992
From: wsc@research.att.com
Date: Wed, 12 Aug 92 18:22 EDT
To: s-news@utstat.toronto.edu
Subject: Re: Modeling with loess

> I have a question about the process of uncovering lack of fit in
> a loess model. To fix ideas, consider the ethanol example that appears
> on page 337 of the Statistical Models on S book. An initial loess surface
> is fit with a span of .5 and degree 2. The (marginal) residuals are then
> fit (with scatter.smooth which has a loess engine) in turn to the two
> explanatory variables and the results are plotted in the top two panels
> of Figure 8.13, page 338. The plot for  the E variable first increases
> and then decreases and the authors conclude that "Clearly there is a lack
> of fit in the right [E] panel", p.337. Refitting the surface with a
> smaller span value leads to a new plot with a similar shape but less 
> variation.
> My approach to diagnosing lack of fit is more mechanical. I have added
> 99% pointwise confidence intervals to the loess smooths of the residuals
> on the explanatory variables. These intervals always include 0 and I am
> tempted to conclude that the original model displayed no lack of fit. My
> question is: How does one move from the mechanical input to good
> judgement in these cases?
> 
> Barry Smith
> Department of Economics 
> York University, Toronto, Canada.

Graphical diagnosis of surplus and lack of fit derives its power from
removing the shackles of mechanical, or formal, analyses.
Associated with the fitting of a model is a body of knowledge about the
mechanism generating the data, the class of models fitted, and the
method of fitting.  The ethanol data are one example. We know that the
method of fitting, local regression, is flexible but can fail to fit
when the curvature in the surface becomes too great, even with locally
quadratic fitting. We can see from the exploratory graphics that there
is a ridge centered at E = 0.9. So as a function of E for fixed C, the
curvature is the greatest at E = 0.9 where a peak occurs. When we see
lack of fit in the residuals, with a peak at E = 0.9, it seems a pretty
sure bet that loess is not quite doing the job. However, that the magnitude
of the distortion is small, say about 0.1 for a surface with a range
of about 4.

True, this looking at the data runs risks of making mistakes. Reacting
to all perceived effects is certainly foolish. But to ignore our large
body of knowledge and only mechanically follow formal statistical
methods in judging effects on diagnostic plots is foolish as well.
There is no replacement for informed judgement.

This does not say that formal methods have no place at all.  But if we
are to use them, they need to have good power, to use a formal term.
Looking at confidence intervals about curves does not have particularly
good power for a global effect that is everywhere significant but
nowhere big. It does better for a very local effect that is big.  For
the former, anova is better. In fact, the model with span = 1 is
strongly rejected vs. span = 1/4 using anova:
> anova(ethanol.first,ethanol.m)
Model 1:
loess(formula = NOx ~ C * E, span = 1/2, degree = 2)
Model 2:
loess(formula = NOx ~ C * E, span = 1/4, degree = 2)
Analysis of Variance Table
      ENP     RSS     Test     F Value      Pr(F) 
1    13.0  4.7372   1 vs 2        7.85 2.4599e-08
2    21.6  1.7999                                


 William S. Cleveland          wsc@research.att.com
 AT&T Bell Labs, Room 2C-286   office: (908)582-6861
 600 Mountain Avenue           fax:    (908)582-3340
 Murray Hill, NJ  07974
Newsgroups: aus.stats.s