Newsgroups: bit.listserv.stat-l
Date:         Tue, 21 Jul 1992 09:23:00 EST
From:         "Philip Gallagher,(919)966-7275" <UPHILG@UNC.BITNET>
Subject:      Power Analysis texts compendium
X-To:         STAT-L@mcgill1.BITNET, st8330@siucvmb.BITNET

    Last week I requested advice on texts to study in order to
    become a Power Guru.  I have had several references to
    established texts and a set of updates on what I suppose to
    be the cutting edge of power research and theory, mostly
    from the shops of Keith Muller at UNC and Ralph O'Brien at
    Florida State.  (Apologies to Ralph O'Brien for having misnamed
    him in my first note.)

    Please note how much brand new power software is being offered
    for distribution in one form or another.  If the major stat
    packages can bring themselves to ignore this, they deserve
    to go down the drain.

    I note that I have heard from no one from the school of thought
    that says that power analyses are totally without value because
    the distributional requirements so vital to power calculations
    are never (well, almost never) satisfied except in stat textbook
    classroom by-hand problems.  (I have certainly not done justice
    to this argument, and I beg anyone who can make it better to please
    do so.)  I know that persons who feel this way exist (there is
    one in our department), and, if you have never run into these
    ideas, it is only fair of me to tell you that they make a strong
    and impassioned case.

    I have had occasion to go back to Cohen's work since I wrote
    first, and I find it to be more useful than my first message
    conveyed.  It is of only limited help with the Muller/O'Brien
    field, but much of their work has reached the literature since
    his 2nd edition was published, so it's only fair to give him
    a break.

    Now I will cut and paste - if I do anyone an injustice, please
    write an explanatory rejoinder directly to STAT-L.

    Thanks to everyone - I now have lots of EXCITING reading material
    for my vacation!


========================================================================
From: phil@wubios.wustl.edu(J. Philip Miller)
I started with Cohn's book but it is too simplified for me as well - he really
does not ever mention the noncentral stat distributions, and that is all you
really have to know if you are prepared to compute things themselves.  I have
worried a lot about this and and one time really wrote some good code to "do
it right", but have become less anamored with it since most of the use of
power that I have on a day to day basis starts with some "off the top of my
head" numbers and to then worry about whether the math is being done "right"
or not is not placing the emphasis where it should.

========================================================================
From:         Ken Hardy <KENHARDY@UNCVM1>

A piece of software that performs many power calculations and has a nice
bibliography on power analyses for various statistics is  PASS (Power
Analysis and Sample Size). I've got the software and will campus mail
you the references section.
========================================================================
From: "dick darlington" <dick_darlington@qmrelay.mail.cornell.edu>

Power analysis is of course highly dependent on the test you're using, so I
will answer just for my specialty: regression and linear models.  Two books
with fairly extensive discussions of power are:
  Rao, C.R. Linear statistical inference and its applications
  Graybill, F.A. Intro to linear statistical models
I should mention that the power tables for linear models in Cohen are not
exact, and do not distinguish between fixed and random scores, which is
necessary for exact power analysis.  My own book "Regression and Linear Models"
gives short BASIC programs that yield exact power values for linear-model tests
under the standard assumptions.  There was a good article on linear-model power
analysis in the Psych Bulletin a year or two ago, but I don't have the
reference handy.
========================================================================
From:     WKJ <JAMES@XAVIER>

Michael Korda's Book " Power " is a lot of fun.

========================================================================
From: jones@reed.edu(Albyn Jones)

well, you could always start with Johnson & Kotz, the set of books
called "Continuous univariate distributions" and look at the non-central
t, non-central chi-squared, non-central F distributions...

========================================================================
From:    "Dennis Roberts" <DMR@PSUVM>

If you want, I will be glad to send you a section from a book of mine re:
power, pretty elementary but may help.  In addition, I would suggest you
contant Jack Barnette at the Univ of Alabama who has an excellent demo
software program on power, that is very useful for "seeing" what power is.
His address is Behavioral Studies, Univ of Alabama, Tuscaloosa Alabama and I
am blocking on the zip.  His bitnet is JBARNETT at UAVM1 ....  but he may
not respond.  If you want my material, send me your surface address and I
will ship immediately.
========================================================================
From:   tgee@ccs.carleton.ca(Travis Gee)

You might try Wm. Hays and Robert Winkler's _Statistics: Probability,
Inference and Decision._ My edition's well out-of-date, but there was
a healthy discussion of it in 1971, at least.  More recently, an
author to search in the PsychLit database is Bruno Zumbo (alas, no
references available in my filing cabinet at the moment). He's a
specialist on relative efficiency of nonparametric stats. His PhD
comprised a Monte Carlo simulation which showed that under violation
of assumptions, nonparametric stats are often _more_ powerful than
their parametric analogues.  If that fascinates you, and you can't
access his papers, you could e-mail him at zumbo@acadvm1.uottawa.ca
He _is_ a power guru.
  Also along those lines, Seigel and Castellan's _Nonparametric
Statistics for the Behavioral Sciences_ discusses the power of various
and sundry methods.  If you're looking at multivariate models, I seem
to recall that the SPSSx manuals and the SAS manuals have references
in many sections which refer to the power of assorted statistics. But
don't try those papers until the groundwork is done.  Good luck!!

========================================================================
From: phil@wubios.wustl.edu(J. Philip Miller)

A recent reference for the simple stuff:

Jurgen Bock & H Toutenburg, Sample Size Determination in Clinical Research,
Chapter 16 in Rao & Chakraborty, eds, Handbook of statistics, vol 8, 515-38,
1991.

========================================================================
From: dixon@srel.edu

Phil:
  Scheffe, The Analysis of Variance, is still a good place to go
for the theory of power for ANOVA.
  Otherwise, the literature on power is really scattered.  There are
lots of 'warm fuzzy' treatments for nonstatisticians, e.g. Cohen, as
you have discovered.  One I give to my consultees is Lipsey, Design
Sensitivity: Statistical power for experimental research, Sage.  It's
a bit more concise than Cohen, but it still shoehorns everything into
an ANOVA framework.
  My favorite source is the Russ Lenth's manual for his PowerPack
program.  It includes a practical guide to power calculations that
lays everything out in a very understandable way, if you know about
non-centrality parameters.  I otherwords, I use it, but I don't expect
my consultees to read it.  PowerPack is also a very easy to use program
that got high marks in Goldstein's review of packages to compute power
(Am. Stat. 43:253).
========================================================================
From:         Jerry Dallal <jerry@NUTMEG.HNRC.TUFTS.EDU>

Have a look at Odeh and Fox, Sample Size Choice, for starters.  Don't buy
it without having a look at it first.  It's very expensive and mostly charts,
but the discussion in the front of the book is reasonable.

I've found Cohen's Statistical Power Analysis for the Behavioral Sciences hard
to read.  It may just be me.  But if you find it hard going, don't think all
texts are like it.  (Who knows?  You may like Cohen and dislike O&F!)
========================================================================
From: Fred Detwiler                                  ST8330   at SIUCVMB

Lipsey, M. W.  (1990).  Design Sensitivity.  Sage.


========================================================================
From: "GEOFF SELIG - CONCORDIA U. COMPUTING SERVICES"
 <ILFC594@Vax2.Concordia.CA>

You might want to have a look at a text by Neter, Wasserman and Kutner
(_Applied Linear Statistical Models, Third Edition_ [Irwin]) for some
fairly good discussions on power (and linear models in general).  As well,
David Howell's _Statistical Methods for Psychology, Third Edition_
[PWS-Kent] has excellent discusions of power in a variety of t-test and
ANOVA situations (and just about everything else).

========================================================================
From: Ralph G. O'Brien <robrien@stat.ufl.edu>

Phil: Bob Parker (Vandy) relayed a memo that you are interested in
becoming more "power"ful. I have just finished the final, final draft of
a long chapter on power analysis (O'Brien, R. G. and Muller, K. E.
(early 1993), Unified Power Analysis for t Tests Through Multivariate
Hypotheses, (to appear) in Applied Analysis of Variance in Behavioral
Sciences, eds. L. K. Edwards, New York: Marcel Dekker.).  We think our
approach is more general and unifying, yet no harder to understand and
apply than Cohen's.  We cover some things he doesn't, and vice-versa.
We don't use constructs such as "small," medium," and "large" effect
sizes as this is an unjustified, oversimplified notion: Most content
researchers will quickly opt for "medium," even though they have no idea
what "medium" implies with respect to their proposed study.  What is a
"small" effect size in one field of study may be a "huge" one in
another.

We have SAS %INCLUDE modules to get things done, available for the
taking via anonymous ftp from my workstation, as outlined below.  Also
included below is an example that should be almost self-explanatory if
you like the cell-means model for ANOVA. The chapter and software
start at a more basic level and go through the general univariate and
multivariate linear models.  We use the modules all the time here in
developing protocols within the Clinical Research Center, and I keep
expanding their capabilities as needs arise.  Our beta version has
options covering tests of correlations (via Fisher's r-to-z), and tests
comparing independent logits from 2 x NumGroups contingency tables.

I am doing a poster paper on power for the multivariate GLM in August at the
Boston Joint Stat Meetings (O'Brien, R.  G. and Shieh, G., "Pragmatic,
Unifying Algorithm Gives Power Probabilities for Common F Tests of the
Multivariate General Linear Hypothesis").  We think our algorithm offers
an "upgrade" on the Muller-Peterson (1984) method, which will be
discussed again in a forthcoming (Dec. 1992?) JASA paper by Muller and
others.  "O'Brien and Shieh" is now being reviewed informally by Jon
Shuster of our group, and it should be submitted soon for journal review.
It will NOT appear in any of the ASA Proceedings. No public software is
being planned/developed yet for the our proposed algorithm.  Actually,
Keith Muller's SAS IML module can be altered in minor ways to do it, but
such a change would be up to Keith, of course.




            -----------------------------------------------
                    INPUT FOR 2 x 3 UNBALANCED ANOVA
            (runs in "base" SAS environment; no IML needed)
             -----------------------------------------------
options ls=72 nosource2;
%include "~robrien/POWER.public/OneWyPow.beta.sas";  *(UNIX format);
* %include OneWyPow;   *(CMS format);
title1 "Example: Unbalanced 2 x 3 ANOVA";
title2 "Order: A1B1 A1B2 A1B3 A2B1 A2B2 A2B3";
cards;
mu 100 110 90 105 130 80 .
weight .2 .2 .1 .2 .1 .2 .
sigma 25 40 .
alpha .05 .01 .
Ntotal 100 120 160 .
nooverall
contrasts
"A Main Effect (Type III)"  1  1  1 -1 -1 -1 .
"B Main Effect (Type III)"  1 -1  0  1 -1  0 .
>                           0  1 -1  0  1 -1 .
"AB Interaction" 1 -1  0 -1  1  0 .
>                0  1 -1  0 -1  1 .
"A2B1&2 vs. A2B3" 0 0 0 .5 .5  -1 .
end
%include "~robrien/POWER.public/FPowTab2.sas";  *(UNIX format);
*  %include FPowTab2;  * (CMS format);

                                --------------
                                PART OF OUTPUT
                                --------------
ALPHA 0.05
       ----------------------------------------------------------
       |                          |           Std Dev           |
       |                          |-----------------------------|
       |                          |      25      |      40      |
       |                          |--------------+--------------|
       |                          |   Total N    |   Total N    |
       |                          |--------------+--------------|
       |                          |100 |120 |160 |100 |120 |160 |
       |                          |----+----+----+----+----+----|
       |                          |Pow-|Pow-|Pow-|Pow-|Pow-|Pow-|
       |                          | er | er | er | er | er | er |
       |--------------------------+----+----+----+----+----+----|
       |EFFCTITL    |TESTTYPE     |    |    |    |    |    |    |
       |------------+-------------|    |    |    |    |    |    |
       |A Main      |2-tailed t   |.156|.178|.222|.090|.099|.116|
       |Effect (Type|-------------+----+----+----+----+----+----|
       |III)        |1-tailed t   |.241|.270|.326|.145|.159|.185|
       |------------+-------------+----+----+----+----+----+----|
       |B Main      |Regular F    |    |    |    |    |    |    |
       |Effect (Type|             |    |    |    |    |    |    |
       |III)        |             |.997|.999|.999|.811|.881|.957|
       |------------+-------------+----+----+----+----+----+----|
       |AB          |Regular F    |    |    |    |    |    |    |
       |Interaction |             |.474|.554|.690|.208|.244|.316|
       |------------+-------------+----+----+----+----+----+----|
       |A2B1&2 vs.  |2-tailed t   |.999|.999|.999|.880|.931|.979|
       |A2B3        |-------------+----+----+----+----+----+----|
       |            |1-tailed t   |.999|.999|.999|.933|.965|.991|
       ----------------------------------------------------------


       -------------------------------------------------------------
       Getting SAS-based Power Shareware Modules Using anonymous ftp
       -------------------------------------------------------------

The best way to obtain the current versions of these modules and
related files is to use the Internet network to ftp-get the files.
ftp (lower case letters may be required) stands for File Transfer
Protocol and is now almost universally available.  The details
regarding where and how these files are stored will change over time.
Accordingly, the instructions given here only tell how to get a short
file that gives complete up-to-date information on this.

(1)  To get to the "front door" of the correct workstation in the
     University of Florida's Department of Statistics enter
               ftp banana.stat.ufl.edu

(2)  To login, use the name
               anonymous

     For the password, enter your full system name, something like
               mmouse@wdw.orlando.fl

(3)  To get a copy of the short information file, enter
               cd pub
               get ReadMe.power myfile.asc
     This will put the contents of a file called ReadMe.power on your
     computer using the file name of your choice ('myfile.asc').

(4)  Enter
               quit
     to return to your local computer, where you can study
     the instructions in ReadMe.power and proceed.

(5)  This method should work for a number of years, but computing
     systems have unpredictable half-lives. If you encounter problems,
     contact me.

 Ralph G. O'Brien, PhD
 Division of Biostatistics              UF Office:   904-392-8446
 Box 100212                             Home office: 904-378-7381
 Univ. of Florida Health Sci. Center    UF fax:      904-392-4168
 Gainesville, FL  32610-0212            email:       robrien@stat.ufl.edu



=======================================================================

Note from Phil Gallagher:  Keith Muller gave me a photocopy of his
paper that Ralph O'Brien mentioned, and it does indeed have "JASA,
December 1992" pencilled on the top.  What a great Christmas present
to look forward to!  (Yes, some of us really that nerdy, Virginia.)
Comments: Gated by NETNEWS@AUVM.AMERICAN.EDU