From: maj@waikato.ac.nz
Newsgroups: bit.listserv.edstat-l
Subject: Re: CI's, conundrums, etc.
Date: 16 Oct 92 11:08:07 +1300
Organization: University of Waikato, Hamilton, New Zealand

In article <m0mfDXk-0003CzC@romulus.reed.edu>, Albyn Jones <jones@REED.EDU> writes:
> Murray A. Jorgensen [ maj@waikato.ac.nz ] writes:
> 
>>My case is that the amount of time and effort needed to
>>communicate the precise meaning of the frequentist confidence
>>machinery can lead to the impression that the technical details
>>are the important part of statistics. They aren't.
>>
[my stuff deleted]
> 
> Sounds pretty reasonable to me.  I don't spend a lot of time on "clever
> conundrums", especially not with the goal of advancing one school of
> inference over another, I'm mostly a data analyst rather than a
> theoretician.  In introductory courses for non-math majors I hardly
> ever mention Bayes methods at all, other than as an aside after
> introducing Bayes' Rule. (Yes, I think students should do a little
> probability along the way to doing statistics.) I do spend lots of
> time on the process of translating substantive problems into statistical
> models, data analysis, and translation back to substantive conclusions.
> We seem to diverge mainly at the last step of your sequence, precisely
> the point where you ask the question "what does it mean?"  Its easy to
> talk nonsense at this point of the process if you don't understand the
> underlying logic.
> 
I would prefer, though, that the discussion be in terms of realistic
examples, though, rather than in terms of TV contests and other
artificial situations.

> I reviewed a paper a few years ago written for a public policy journal
> advocating a change in a federal rule.  The empirical question was one
> of risk, how dangerous consumption of some possibly toxic quasi-food
> item was.  The paper reviewed 10 studies, each of which reported a
> significance test of the null hypothesis of "no risk", more precisely
> no difference in risk between the experimental group and the control
> group.  The argument presented in the paper came down to precisely the
> claim that repeated failures to reject the null hypothesis constitute
> strong evidence that the null hypothesis is true.  The estimated
> effect sizes were non-trivial, and in the same direction, but all the
> studies were done with small sample sizes.  Looking at the paper, I
> came to the opposite conclusion, namely the paper actually presented
> very strong evidence of a non-trivial risk.  The author had a little
> trouble translating from the statistical formalism back to substantive
> conclusions.
>

An excellent example! I must try to include something similar
into part of my course.
 
> Agreed, this is not an example of damage done by
> misunderstanding what a 95% confidence interval means.  It is an
> example of the more general problem of failing to understand the
> basic logic of statistical inference.  I use this example every year,
> and it makes some students uncomfortable, because they come to the course
> with the belief that statistics is a means of getting "the answer",
> and any hint that they might have to think a bit is scary.  Most
> students react very positively to such examples.  I don't see it as
> a waste of precious time better spent talking about modeling, because
> it is a part of talking about the translation back to the substantive
> area.
> 
> albyn jones
> jones@reed.edu

Just as an aside, though, you would draw much the same conclusions
from your example whether the intervals in question were
confidence intervals or Bayesian highest posterior density intervals.

I'm not trying to buy a fight, though, I accept that the duty of
an instructor in introductory statistics is to teach the logic of
classical frequentist inference with all the clarity s/he can
muster.
-- 
Murray A. Jorgensen [ maj@waikato.ac.nz ]    University of Waikato
Department of Mathematics and Statistics     Hamilton, New Zealand      
__________________________________________________________________
'Tis the song of the Jubjub! the proof is complete,
if only I've stated it thrice.'