HDTenEightyP/NetNews huggingface dataset:
From: cfk6u@fermi.clas.Virginia.EDU (Charles F. Kramer)
Subject: 'Data visualization' software
Organization: University of Virginia
Distribution: usa
Date: Fri, 18 Dec 1992 19:09:40 GMT

This article was probably generated by a buggy news reader.
Newsgroups: sci.math.stat
From: thompson@atlas.socsci.umn.edu (T. Scott Thompson)
Subject: Re: stepwise regression
Reply-To: thompson@atlas.socsci.umn.edu
Organization: Economics Department, University of Minnesota
Date: Thu, 17 Dec 1992 19:44:00 GMT

feh@duke.edu (Frank Harrell) writes:

>Backward elimination does not at all dominate full model fitting.
>See:

[references deleted.]

I am not quite sure what you mean by "full model fitting".  (I can't
check your references right away to find out, since here at Minnesota
the various statistical journals are scattered across two cities, and
I would have to visit possibly three libraries to find them all!)

I will take it to mean "using everything", i.e. running OLS on the set
of all possible regressors, dealing with the colinearities that arise
in some appropriate fashion.  I hope this isn't far off.

I looked again at "Introduction to the Theory and Practice of
Econometrics" (Judge et. al.) that I mentioned in my earlier post.
You are correct that backwards elimination (a form of pretest
estimator) does not dominate the "use everything" strategy.  However,
full model fitting does not dominate backwards elimination either,
provided the "elimination criteria" in the latter is chosen
appropriately.  Neither dominates the other.

I wouldn't want to push this too hard, since the default elimination
criteria implemented in most software is unlikely to be appropriate
except by coincidence.  As I said originally, I mostly sympathize with
the opinion that stepwise regression is "the work of the Devil."

The same chapter (20) also compares shrinkage estimators to the "full
model" approach, and cites evidence that a "Stein-like" shrinkage
estimator does dominate "full model fitting," rendering the latter
inadmissible. It is not clear from this book whether or not backwards
elimination is also dominated by the shrinkage estimator.

(I have always thought of pretest estimators as a randomized form of
shrinkage estimation, hence my confusion here.)

The textbook cites as a source for proofs of some of these assertions:

Judge, G.G., and M.E. Bock (1978) *The Statistical Implications of
Pre-Test and Stein-Rule Estimators in Econometrics*.  Amsterdam:
North-Holland.

I don't think that we can resolve these issues very well unless we get
more specific about loss functions.  I haven't looked at the work by
Judge and Bock for several years, but I believe that it concentrates
on quadratic (in the regression parameters) loss, with the usual
minimum expected loss as the criterion.  Perhaps the results are
different when the loss function is quadratic in some measure of
predictive performance instead.  It is quite conceivable to me that
the ranking of various procedures could be reversed by choosing one or
the other risk criterion.

My main point stands, however.  An estimator is simply a function of
the data.  It is neither good nor bad, "automatic" or "thoughtful" on
its own.  It is how we use them in a given situation that counts.
--
T. Scott Thompson              email:  thompson@atlas.socsci.umn.edu
Department of Economics        phone:  (612) 625-0119
University of Minnesota        fax:    (612) 624-0209
Newsgroups: sci.math.stat
From: rons@hardy.u.washington.edu (Ronald Schoenberg)
Subject: Re: Binary Correlations. Was: Levels of Measurement?
Organization: University of Washington, Seattle
Date: Thu, 17 Dec 1992 20:36:37 GMT

In article <thompson.724607838@daphne.socsci.umn.edu> thompson@atlas.socsci.umn.edu writes:
>Perhaps someone interested in the levels of measurement question might
>be able to help me too.
>
>I have been acting as informal statistical consultant for my wife, who
>is a physician with only an elementary statistical background, in a
>project where she wants measures of association between a fairly large
>number of binary variables in a sample with n=500 and in a subsample
>with n = 260.  (These numbers may increase in the future.)  
>
>We have been calculating standard Pearson correlation coefficients,
>which to my way of thinking are as good as anything else at describing
>binary associations, since scaling is not an issue for binary data.  I
>
[....]
>
>If you were in my shoes would you stick with the Pearson correlations
>or do something different?  If the latter, would you provide a
>reference please.
>
>Related and more interesting question:
>
>What is "the right way" to do this from the point of view of
>statistical theory?
>--

I would compute polychoric coefficients - conceptually they are
correlations of latent normal variables for which below a threshold we
observe the case being in one category and above we observe it in
another category.
Newsgroups: sci.math.stat
From: pmk@busadm1.cba.hawaii.edu (PAUL KAZENSKI)
Subject: 2 vs 1 system of linear equations
Organization: College Business Administration, University of Hawaii
Date: Wed, 16 Dec 1992 03:36:05 GMT

I've received a question I cannot provide a good answer to:

There are 2 systems of equations comprised of 3 equations each that
have been estimated recursively with "good" results;  it is being
suggested that these be combined into a single system and estimated 
simultaneously in order to capture additional information from 
(possibly) correleated error terms between the two systems.

Since the task is non-trivial, the questioner asks if there are tests
which can be performed to determine **if** it is necessary to combine
systems.  He has tried correlation analysis of the residuals--which 
indicates no correlation--and regressed error terms against all others
with the same results.

If anyone knows of tests/references, I would appreciate either your post or 
e-mail.  I will pass it on and summarize to the net.   

Mahalo Nui Loa!
Newsgroups: sci.math.stat
From: luke@cs.city.ac.uk (Luke Whitaker)
Subject: Re: Statistical software for Macintosh ?
Organization: City University
Date: 18 Dec 92 13:24:51 GMT

In <1992Dec17.144106.7015@wega.rz.uni-ulm.de> POKORNY@rzmain.rz.uni-ulm.de (POKORNY DAN) writes:

>Statistical software for Mac - who can advise?
>We are searching for a good statistical software for Macintosh.

I have played around with a demo version of the JMP package for
the Mac, and I was quite impressed. It is produced by SAS institute, but
is not the same as the SAS package as seen on a variety of other platforms.

JMP is menu driven, the stats appeared to be qood, as did the graphics,
particularly for data exploration perhaps rather than presentation.

I stress that I have done no more than 'play' with a demo version. It was
quite expensive (the main reason I don't actually have a real copy).

Luke
From: thompson@atlas.socsci.umn.edu (T. Scott Thompson)
Newsgroups: sci.math.stat
Subject: Binary Correlations. Was: Levels of Measurement?
Date: 17 Dec 92 15:57:18 GMT
Reply-To: thompson@atlas.socsci.umn.edu
Organization: Economics Department, University of Minnesota

Perhaps someone interested in the levels of measurement question might
be able to help me too.

I have been acting as informal statistical consultant for my wife, who
is a physician with only an elementary statistical background, in a
project where she wants measures of association between a fairly large
number of binary variables in a sample with n=500 and in a subsample
with n = 260.  (These numbers may increase in the future.)  

We have been calculating standard Pearson correlation coefficients,
which to my way of thinking are as good as anything else at describing
binary associations, since scaling is not an issue for binary data.  I
do not claim to have studied the issue in depth, however.

I am an econometrician and I know that different fields have different
statistical subcultures and norms, so I am not sure that this is what
other physicians, biostatisticians, epidemiologists, etc. expect.  My
impression is that medicine is the field where statistical practice is
most codified, meaning that there are conventions that must be
satisfied if you want to get published, even if they don't make a
whole lot of sense in the particular application.

So here is my question:

If you were in my shoes would you stick with the Pearson correlations
or do something different?  If the latter, would you provide a
reference please.

Related and more interesting question:

What is "the right way" to do this from the point of view of
statistical theory?
--
T. Scott Thompson              email:  thompson@atlas.socsci.umn.edu
Department of Economics        phone:  (612) 625-0119
University of Minnesota        fax:    (612) 624-0209
Newsgroups: sci.math.stat
From: mk9y@faraday.clas.Virginia.EDU (Michael  Kubovy)
Subject: Re: ROC?
Organization: University of Virginia
Date: Sat, 19 Dec 1992 14:57:37 GMT

charlie@umnstat.stat.umn.edu (Charles Geyer) writes:
>>dan@krider.concord.wvnet.edu (Dan Krider) writes:
>>Receiver operating characteristic (ROC) curves are a part of decision  
>>theory and are used in medical science. Can anyone tell me why it is  
>>called this (ROC)?
>
>Comes from electrical engineering I believe.  "receiver" means a radio
>reciever.  I'm not sure exactly what is plotted against what

Hit rate [p("yes"|signal)] against false-alarm rate [p("yes"|noise)]
when (1) the payoffs for hits, correct rejections, misses, and false
alarms are varied, or (2) the prior probabilities of signal and noise
are manipulated. In psychophysics we are trying to standardize the terminology
to "isosensitivity function."
--
|\  /| / MICHAEL KUBOVY (kubovy@virginia.edu)
| \/ |/		OFFICE: 804-982-4729; LAB: 982-4751; FAX: 982-4766
|    |\ 	   Department of Psychology, Gilmer Hall,
|    | \ 	   Univ. of Virginia, Charlottesville, VA 22903-2477
Newsgroups: sci.math.stat
From: charlie@umnstat.stat.umn.edu (Charles Geyer)
Subject: Re: ROC?
Organization: School of Statistics, University of Minnesota
Date: Sat, 19 Dec 1992 04:53:03 GMT

In article <BzGupn.22x@news.concord.wvnet.edu> dan@krider.concord.wvnet.edu (Dan Krider) writes:
>Receiver operating characteristic (ROC) curves are a part of decision  
>theory and are used in medical science. Can anyone tell me why it is  
>called this (ROC)?

Comes from electrical engineering I believe.  "receiver" means a radio
reciever.  I'm not sure exactly what is plotted against what, but the
point is the same.  If you turn up the sensitivity, you amplify the noise.

-- 
Charles Geyer
School of Statistics
University of Minnesota
charlie@umnstat.stat.umn.edu
Organization: University of Illinois at Chicago
Date: Wed, 16 Dec 1992 20:15:18 CST
From: <U53076@uicvm.uic.edu>
Newsgroups: sci.math.stat
Subject: Levels of Measurement?

Organization: University of Illinois at Chicago
Date: Wed, 16 Dec 1992 19:47:58 CST
From: <U53076@uicvm.uic.edu>
Newsgroups: sci.math.stat
Subject: Levels of measurement

Hi,
   Let me first apologize for boring you with such a mundane question.   Anyway
I am in a dispute about levels of measurement and the appropriate statistics to
use when dealing with data at the ordinal level.  I'd like to compare performan
ce on a communication task across two different contexts.  In both cases,
competence is measured on an ordinal scale, for simplicity's sake, 1, 2, 3.  It
has been my practice in the past to compute a Pearson product-moment correlatio
n between the two tasks and to use the resulting r as an indication of the
degree to which performance is consistent across tasks.  I recently was told
that this was an inappropriate test cause I was only at the ordinal level.  But
my references (see esp. Glass & Hopkins, 1984; Heermann & Braskamp, 1970) say
that the whole levels of measurement question has ben blown out of proportion,
and that the sort of usage I describe is just fine.  What do you think?

Glass, G. V., & Hopkins, K. D. (1984). Statistical methods in education and
    psychology.  Englewood Cliffs, NJ: Prentice Hall.
Heermann, E. F., & Braskamp, L. A. (1970). Readings in statistics for the
    behavioral sciences.  Englewood Cliffs, NJ: Prentice Hall.


Thanks in advance.  If I'm wrong about this, I'd like to change.  If I'm right,
I'd like to rub it in. . .  :-)

                               Bruce Lambert
                               Assistant Professor
                               Pharmacy Administration
                               U. of Illinois @ Chicago
Newsgroups: sci.math.stat
From: charvey@teal.csn.org (Chip Harvey)
Subject: Text references for distributions
Organization: Colorado SuperNet, Inc.
X-Newsreader: Tin 1.1 PL4
Date: Thu, 17 Dec 1992 23:46:04 GMT

Is Johnson & Kotz still considered to be the best reference for distributions?
The Houghton-Mifflin series by Johnson & Kotz appears good, but perhaps there
are better or more recent and less expensive references.  I'd appreciate
any such information.  Thanks in advance & Season's Greetings.



Chip Harvey
charvey@csn.org
From: bmann@desire.wright.edu
Newsgroups: sci.math.stat
Subject: Consulting Centers talks
Date: 15 Dec 92 12:31:57 EST
Organization:  Wright State University 

I am trying to organize a special contributed papers session for the
Statistical Consulting Section of the ASA to take place during the
meetings in San Francisco next August.  The general topic is "Models
of Statistical Consulting Centers in Universities"  or something like
that.  I am especially looking for people who would contribute a description
of a center that is associated with a School of Medicine or a School of
Business to round out the variety of papers.  If you are such a person
or if you have a suggestion of someone who you think would be good, please
send me an email message - bmann@desire.wright.edu or bmann@wsu.bitnet.

Thanks in advance. - Barbara L. Mann, Dept. of Math & Stat, Wright State U,
Dayton, OH 45435.
From: thompson@atlas.socsci.umn.edu (T. Scott Thompson)
Newsgroups: sci.math.stat
Subject: Re: A Simple Question about Instrumental Variables
Date: 16 Dec 92 17:22:59 GMT
Article-I.D.: kiyotaki.thompson.724526579
Reply-To: thompson@atlas.socsci.umn.edu
Organization: Economics Department, University of Minnesota

bubai@matt.ksu.ksu.edu (Priyabroto) writes:

>Hi,

>I had a small question:

>Let Y = Xb + u   (in matrix notation)

>such that Cov(X,u) is not equal to zero.

>Now, we use Z as an instrumental variable where:

>Cov(X,Z) is not equal to zero but,
>Cov(Z,u) is equal to zero.

>Then, WHY is it that b hat(instr. var) = (Z'X)inverse * Z'y?

I don't understand the question.  This is the _definition_ of the IV
estimator (assuming that Z and X have the same dimensions).  The
statistical motivation for the estimator comes from the fact that

Inverse(Z'X)*Z'y = b + Inverse(Z'X/n)*(Z'u/n)

and the last term converges to zero in large samples due to the law of
large numbers and the assumption that E(Z'u) = 0.

>As we know b hat = (X'X)inverse * X'y.

This is the definition of the least squares estimator, which is not
the same as the IV estimator, and which does not converge to b in
large samples when X and u are correlated.

Perhaps you are asking how to show that the IV estimator and the least
squares estimator are the same thing.  This cannot be done in general.
The least squares estimator can be thought of as a special case of the
IV estimator obtained by setting Z = X.

--
T. Scott Thompson              email:  thompson@atlas.socsci.umn.edu
Department of Economics        phone:  (612) 625-0119
University of Minnesota        fax:    (612) 624-0209
From: rick@daffodil.tymnet.com
Newsgroups: sci.math.stat
Subject: Re: shortest chain on a sphere / shortest around the world trip
Date: 14 Dec 92 17:21:22 GMT
Organization: BT North America, San Jose, CA

In article <13DEC199216365553@cc.utah.edu>, lallu@cc.utah.edu (Upmanu Lall, Civil Eng) writes:
|> 
|> Does anyone know if this is a solved problem ?
|>  It seems like it ought to be.
|> There are n arbitrary locations on the surface of the earth, each of which must 
|> be visited
|> once and only once, starting from any one of them.
|> At the end of the trip one is back at the starting point.
|> It seems to me that the choice of the starting point does not matter,
|> i.e. there is a unique shortest chain, and you should be able to find it
|> irrespective of where you start on it.
|> Is this assertion correct ?
|> Is there an algorithm for determining the shortest path efficiently, 
|> and the second shortest, etc ?
|> I recall from a network optimization class taken 2 decades ago, that 
|> a chain network is the dual problem to a flow network
|> Can this be used to setup the problem such that one could 
|> establich whether the primal or the dual is easier to solve, and to then solve 
|> the problem ?
|> Distances are Euclidean on the sphere (or the Earth). LAts and Longs of the 
|> points are known , and the number of locations is also known a priori (i.e. 
|> this is not a stochastic process).
|> 
|> Any references, or suggestions to bodies of literature shall be appreciated.
|> Determinitic  or Monte Carlo techniques would be fine.
|> 
|> 
|> Thanks very much for your help
|> Please respond to
|> ulall@pub.uwrl.usu.edu


Try Travelling Salesman Problem in "Introduction to Operations Research" by
Hillier and Liebermann.
Newsgroups: sci.math.stat
From: tdj@infonode.ingr.com (Ted Johnson)
Subject: FAQ for sci.math.stat?
Organization: Intergraph Corporation, Huntsville, AL.
Date: Fri, 11 Dec 1992 19:57:56 GMT


Is there a FAQ (Frequently Asked Questions) file for sci.math.stat?
If so, and if I can get it (by ftp or email), how do do this?
If so, and it's just posted monthly, please let me know and I'll wait.
If not, I'll let your silence be my answer.
Thank you.

Ted Johnson, Intergraph Corp.
tdj@blzbub.b17d.ingr.com or tdj@infonode.ingr.com
Newsgroups: sci.math.stat
From: lallu@cc.utah.edu (Upmanu Lall, Civil Eng)
Subject: shortest chain on a sphere / shortest around the world trip
News-Software: VAX/VMS VNEWS 1.4-b1  
Organization: University of Utah Computer Center
Date: 13 Dec 1992 16:36 MST  


Does anyone know if this is a solved problem ?
 It seems like it ought to be.
There are n arbitrary locations on the surface of the earth, each of which must 
be visited
once and only once, starting from any one of them.
At the end of the trip one is back at the starting point.
It seems to me that the choice of the starting point does not matter,
i.e. there is a unique shortest chain, and you should be able to find it
irrespective of where you start on it.
Is this assertion correct ?
Is there an algorithm for determining the shortest path efficiently, 
and the second shortest, etc ?
I recall from a network optimization class taken 2 decades ago, that 
a chain network is the dual problem to a flow network
Can this be used to setup the problem such that one could 
establich whether the primal or the dual is easier to solve, and to then solve 
the problem ?
Distances are Euclidean on the sphere (or the Earth). LAts and Longs of the 
points are known , and the number of locations is also known a priori (i.e. 
this is not a stochastic process).

Any references, or suggestions to bodies of literature shall be appreciated.
Determinitic  or Monte Carlo techniques would be fine.


Thanks very much for your help
Please respond to
ulall@pub.uwrl.usu.edu
From: bubai@matt.ksu.ksu.edu (Priyabroto)
Newsgroups: sci.math.stat
Subject: A Simple Question about Instrumental Variables
Date: 15 Dec 1992 19:30:38 -0600
Organization: Kansas State University
NNTP-Posting-Host: matt.ksu.ksu.edu

Hi,

I had a small question:

Let Y = Xb + u   (in matrix notation)

such that Cov(X,u) is not equal to zero.

Now, we use Z as an instrumental variable where:

Cov(X,Z) is not equal to zero but,
Cov(Z,u) is equal to zero.

Then, WHY is it that b hat(instr. var) = (Z'X)inverse * Z'y?

As we know b hat = (X'X)inverse * X'y.

Would really appreciate any help.



Newsgroups: sci.math.stat
From: rivas@fraser.sfu.ca (Jorge David Rivas)
Subject: Re: Test for Linearity ??
Organization: Simon Fraser University, Burnaby, B.C., Canada
Date: Fri, 11 Dec 1992 22:02:20 GMT

In article <2975@tymix.Tymnet.COM> rick@daffodil.tymnet.com (Rick McFarland) writes:
>Why don't you try calculating the correlation coefficient: the closer it
>is to 1 or -1, the more linear your data.
>
>
>
>------------------------------------------------
>"ad astra per aspera"

I do not think that the corr. coef. would tell much at all right away.    
If he is asking about linearity I'm sure he has in mind fitting a straight line.
He says:

               " I have a question regarding, linearity
of data points.  Does anybody know of any statistical
test for linearity. I would appreciate any kind of
help, references etc."

I would think that PLOTTING the data first would be necessary in order to asses
the rest of the test:

ie: if there is apattern other than somekind of random scattering along some
invisible straight line (ie: curvilinear like a parabola) then he should fit
a polynomial model (possibly of second order), then the reduced model with only
the linear term and THEN asses linearity by means of the corresponding tests.

If not consider looking at : "Problem solving: a statistician's guide"
                             Christopher Chatfield,
                            Chapman and Hall,    pages 117-118

In it there are 4 sets of data constructed by Anscombe (1973). All have r=.82,
however:

set 1 is your typical "around the straight line" cloud

set 2 is 3/4 of a seemingly "branches down" parabola on which all the point are
      perfectly lined on it.

set 3  is a set of 11 points in which 10 are perfectly lined on a stright line
      and the 11th is an obvious outlier

set 4 is a setof 11 points, but this time the first 10 make a perfectly vertical      straight line. The 11th one is again a far away outlier.

REMEMBER:  ALL them have r = 0.82

   Jorge David Rivas        (undergrad) Dept. of Math & Stats
                                       Simon Fraser University
                                       Burnaby, B.C.  CANADA
From: prasundi@kittyhawk.ecn.uoknor.edu (Pankaj Raghavendra Asundi)
Newsgroups: sci.math.stat
Subject: SPLIT-SPLIT-PLOT LATIN SQUARE CODE NEEDED
Date: 13 Dec 92 18:25:24 GMT
Article-I.D.: constell.Bz7n6C.9y
Distribution: na
Organization: Engineering Computer Network, University of Oklahoma, Norman, OK, USA

Hi Netters;
I have a design that looks like this and belongs to
the category of " SPLIT-SPLIT-PLOT LATIN SQUARE DESIGN".



                Bo   B1    B2
              ---------------
               C0   C1    C2

          S0   3    4    6
   A0     S1   1    4    6
   ---------------------------
               C2   C0   C1
          
          S0   3    4    6
   A1     S1   1    4    6
   --------------------------
               C1   C2  C0
          
          S0   3    4    6
   A2     S1   1    4    6
   ---------------------------



   I would like some help on coding this for SAS.  Unfortunately the SAS manuals
   do not have an example that is similar do this.
   Please post on the net or e-mail to my address.

   Also can somebody help me with this question 
   " why do we initially assume that the null is true rather than
   the alternative?  what is the mathematical reason for this?

   Thanks in advance

   my e-mail address : prasundi@uokmax.ecn.uoknor.edu


          
Newsgroups: sci.math.stat