HDTenEightyP/NetNews huggingface dataset:
From: hrubin@pop.stat.purdue.edu (Herman Rubin)
Newsgroups: sci.math.stat
Subject: Re: simulation of 2-dimensional Poisson distribution
Date: 17 Aug 92 16:04:41 GMT
Organization: Purdue University Statistics Department

In article <1992Aug16.234829.18925@ee.ubc.ca> victorw@ee.ubc.ca (Victor J. K. Wong) writes:

>For a 2-dimensional homogenous Poisson distribution, 
>p(i,A) = exp(-A*d)*(A*d)^i/i! where p(i,A) is the probability
>of i occurrences in an area A, d is the spatial density
>(number of occurrences per unit area). Similar to the
>one-dimensional Poisson distribution, 2-dimensional one
>also has the memoryless property. 

>To simulate a 2-dimensional Poisson environemnt, one can
>divide an area A into many small areas with equal size B.
>If B is small enough, p(0,B) + p(1,B) is very close to 1
>and hence, p(i,B), for i > 1, can be discarded. For each
>small area B, the probability of occurrence is p(1,B) and
>the probability of no occurrence is p(0,B), and this is
>a binominal process. If B is very small, one can assume
>that the location of the occurrence is the center of the 
>area B If this process repeats for each area B, the 
>distribution of of any area in A is a 2-dimensional
>Poisson. However, such a simulation takes long time, does
>anyone has better suggestion for a faster simulation of
>a 2-dimensional Poisson environment? If you have any
>suggestion, please email to me (email address victorw@ee.ubc.ca).

I am posting this response because I believe that there are
misconceptions which seem to be shared by many.  I believe
that what is asked is the simulation of a Poisson process, 
and that what is wanted is the entire set of sample points.
Now if the area of the entire region is A, there are roughly
A*d points to be computed, and there is no way to do the job
without this much computing.  But not much more is needed.

One way to do the job is to first compute the x-coordinates
of the points, which even with the greatest generality from
a one-dimensional Poisson process, with the measures of the
spacings being independent exponential random variables.  
Then the y-coordinates have the uniform (or other appropriate)
distribution given the x-coordinates.  This process can also
be modified to accommodate acceptance-rejection procedures
with little difficulty.

-- 
Herman Rubin, Dept. of Statistics, Purdue Univ., West Lafayette IN47907-1399
Phone: (317)494-6054
hrubin@pop.stat.purdue.edu (Internet, bitnet)  
{purdue,pur-ee}!pop.stat!hrubin(UUCP)
From: sutton@lamar.ColoState.EDU (Rich Sutton)
Newsgroups: sci.math.stat
Subject: Cox & Hinkley FOUND
Summary: still in print
Date: 21 Aug 92 15:47:59 GMT
Followup-To: sci.math.stat
Distribution: usa
Organization: Colorado State University, Fort Collins, CO  80523

Overnight I received three replies informing that my local bookseller
had given me bad advice. Cox & Hinley's Theoretical Statistics is
still in print. I just ordered a copy from Chapman & Hall, the publisher,
at 212-244-6412.

Chapman & Hall informed me that the solution manual went out-of-print this
last January.

Thanks to T.S. Lim, Jose Albert, and Ron Thisted for setting me straight.

-Rich Sutton
From: hrubin@pop.stat.purdue.edu (Herman Rubin)
Newsgroups: sci.math.stat
Subject: Inference  Was: Re: Standard Deviation.
Date: 19 Aug 92 14:57:26 GMT
Organization: Purdue University Statistics Department

In article <1992Aug18.214711.6657@mailhost.ocs.mq.edu.au> wskelly@laurel.ocs.mq.edu.au (William Skelly) writes:
>In article <WVENABLE.92Aug18180002@algona.stats.adelaide.edu.au> wvenable@algona.stats.adelaide.edu.au (Bill Venables) writes:

			...............

>>In my opinion statistical inference is all about reliably capturing
>>information from data (and elsewhere if you are a Bayesian); it's not
>>really about coming up with a number from a data set that you can show will
>>be "close" to an unknown parameter value, in some special sense of "close".

>I am not sure I follow you.  From my applications I only want to test
>some null-hypothesis (perhaps a narrow application...but very useful!).
>Generally I want to know if two samples are from the same population.
>Isn't this just asking whether or not the two sample means are close?

If you want the answer to whether two samples are independent samples
from EXACTLY the same population, I need no data to tell you that the
answer is no.  If you want to know if two samples approximately have
this property, and this is a very important problem, the answer may
very well be yes.  But it is necessary to balance the risks.  

In some cases, a good way to do this may be by looking at the closeness
of the sample means, but in other cases, there may be much better ways.
But how to look at them is much more complicated than you were led to
believe.
-- 
Herman Rubin, Dept. of Statistics, Purdue Univ., West Lafayette IN47907-1399
Phone: (317)494-6054
hrubin@pop.stat.purdue.edu (Internet, bitnet)  
{purdue,pur-ee}!pop.stat!hrubin(UUCP)
Newsgroups: sci.math.stat
From: regeorge@magnus.acs.ohio-state.edu (Robert E George)
Subject: Re: Fwd: Standard Deviation.
Organization: The Ohio State University
Date: Sat, 15 Aug 1992 01:13:53 GMT

In article <1992Aug15.000049.29790@leland.Stanford.EDU> dhinds@leland.Stanford.
EDU (David Hinds) writes:
>In article <1992Aug14.231916.23479@magnus.acs.ohio-state.edu> regeorge@magnus.
acs.ohio-state.edu (Robert E George) writes:
>>
>>More intuitively, if we take a very sample, we are less likely to get
>>extreme values and so our notion of what the population variance is (note
>>that I am not proposing some particular estimator) will be unrealistic.
       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
>>For instance, I give an exam to two students. Their scores are 67 and
>>71. I think, "Gee, there's not a lot of variability in these scores."
>>But then 8 more students take the exam:
>>     60 78  100 38  50 88  99 39
>>
>>and it now is clear there *is* more variability in these scores.
>>
>>But let me reiterate that T will *always* have a negative bias for the
>>population variance whatever the sample size is
>
>This is wrong.  If this set of 8 values is the "population", sure, for
>your particular sample, the predicted variance happens to be less than
>the population variance.  However, the mean variance of all samples
>from this population will equal the population variance (i.e., the
>sample variance is an unbiased estimator).  Note that most samples of
>size 2 from this population will have much larger variances than the
>one you picked, and many will have variances larger than the parent
>population (i.e., you are just as likely to get [100 38] as [67 71]).
[deletions]
  The 8 values are not necessarily the whole population (which may
well be finite).
   For any given range of values, the probability that a RS of size
8 will be contained in that range is no greater than the probability
that a RS of size 2 will be contained in that range.

My intent (as I made clear) wasn't that we think of a particular
estimator, just that we
look at the data and try to see how variable it is.

Robert George
(speaking only for myself)
From: mglacy@lamar.ColoState.EDU
Newsgroups: sci.math.stat
Subject: Adjusted r-squared as analogous to sample variance estimator
Date: 20 Aug 92 21:26:48 GMT
Organization: Colorado State University, Fort Collins, CO  80523

After following the thread about the approporiate divisor for the
sample variance, and the comments about the misconceptions and
mistakes concerning it that are propogated in introductory textbooks,
I was struck by an inconsistency.  While most intro. books make
considerable noise about the need to use n-1 to make the sample
variance an unbiased estimator, virtually none of them consider
the analogous correction to make the sample r-squared an unbiased
estimator of the population parameter.

Does anyone have any explanation for this inconsistency, other
than simple error on the part of textbook writers?
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
Mike Lacy  Dept. of Sociology  Colo. State Univ.   Fort Collins CO 80523
Opinions and truth-claims offered above are mine, not my employer's.
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
Newsgroups: sci.math.stat
From: mpesubra@athena.mit.edu (Velusamy Subramaniam)
Subject: Fuzzy sets
Organization: Massachusetts Institute of Technology
Date: Mon, 17 Aug 1992 14:37:14 GMT

Is there an introductory text on fuzzy sets for the layman?  Thanks.
Newsgroups: sci.math.stat
From: wskelly@laurel.ocs.mq.edu.au (William Skelly)
Subject: Re: Standard Deviation.
Keywords: (n) versus (n-1)
Organization: Macquarie University, Australia.
Date: Sun, 16 Aug 1992 21:11:42 GMT

In article <1992Aug14.172833.11844@cbfsb.cb.att.com> rizzo@cbnewsf.cb.att.com (anthony.r.rizzo) writes:
>Can someone explain why calculating the Standard Deviation (SD),
>for small samples, with (n-1) in the denominator is better than
>doing so with (n) in the denominator?  I'm sure that there's
>a perfectly good reason for doing so.  But we, lowly engineers
>aren't usually told the reason.  Thanks now, for your response later.;-)
>
>Tony Rizzo (att.com!homxc!rizzo)
>

Well the (n-1) term is used when you have a "biased" condition.
I don't know the full theoretical reasons, but if you use "n"
instead of (n-1) your standard deviation is _smaller_. so if you
are less than sure that your sample is truely representative of
your population you use "n-1" which provides for a slightly
larger standard deviation.  Of course when you are dealing with
very small samples (n < 30 maybe) this could make a heck of a
difference in the resulting std. dev. value.

chris
From: mcclella@yertle.Colorado.EDU (Gary McClelland)
Newsgroups: sci.math.stat
Subject: Re: Adjusted r-squared as analogous to sample variance estimator
Date: 22 Aug 92 21:52:59 GMT
Organization: University of Colorado, Boulder

mglacy@lamar.ColoState.EDU writes:

>After following the thread about the approporiate divisor for the
>sample variance, and the comments about the misconceptions and
>mistakes concerning it that are propogated in introductory textbooks,
>I was struck by an inconsistency.  While most intro. books make
>considerable noise about the need to use n-1 to make the sample
>variance an unbiased estimator, virtually none of them consider
>the analogous correction to make the sample r-squared an unbiased
>estimator of the population parameter.

>Does anyone have any explanation for this inconsistency, other
>than simple error on the part of textbook writers?

You just haven't been reading the right books!  Here are two books which
treat the subject of "shrunken" or "adjusted" R-sq.

Cohen, J., & Cohen, P. (1983).  Applied Multiple Regression/Correlation
  Analysis for the Behavioral Sciences. (2nd ed.)  Lawrence Erlbaum.
  (see pp 105-107).

Judd, C.M., & McClelland, G.H. (1989).  Data Analysis: A Model Comparison
  Approach.  HBJ.  (see Chapter 5; this book incidentally gives a much
  more coherent, intuitive explanation of both n-1 and the R-sq adjustment
  without ever using the phrase "degrees of freedom.")

gary mcclelland
univ of colorado
mcclella@yertle.colorado.edu
From: thompson@atlas.socsci.umn.edu (T. Scott Thompson)
Newsgroups: sci.math.stat
Subject: Degrees of Freedom  Was: Re: Standard Deviation.
Date: 20 Aug 92 19:19:57 GMT
Reply-To: thompson@atlas.socsci.umn.edu
Organization: Economics Dept., University of Minnesota

hougen@uirvlh.csl.uiuc.edu (Darrell Roy Hougen) writes:

>Before people get too confused, just let me add my two cents worth.
>The number of "degrees of freedom" in a problem represents the amount
>of information in the data and is generally equal to the number of
>samples.  Intuitively, as statistics are computed using the data, the
>independent information in the data is gradually "used up" and the
>number of degrees of freedom is reduced.  The information is not
>actually lost, but the statistics that have been computed contain some
>of the information so that the amount of independent information left
>in the data is reduced.

>[example of sample mean deleted]

I have never been very satisfied with this notion of "degrees of
freedom" being equivalent to "pieces of independent information".  The
correspondence works well enough for linear models and independent
observations, as in the example given, but there are alternative
interpretations (in terms of the dimension of orthogonal subspaces)
that do just as well in this setting.

To put out a concrete example that illustrates the nature of my
qualms, consider the following.  Suppose my data consist of a single
number, say X.  Suppose that I am given two pieces of information
about X (but not X itself):

           (1)  I can observe Y := X**2
           (2)  I can observe Z := {  1 if X > 0
                                   { -1    otherwise

How many "pieces of information" do I now have?  Clearly I can recover
X from Y and Z by the formula X = Z * <positive square root of Y>, but
neither Y nor Z alone suffices (excluding the special case where 
X = Y = 0).  Thus Y and Z must each "use up" (at least) one degree of
freedom in the usual accounting.  Thus I must conclude that

       <degrees of freedom in X> .GE. 1 + 1 = 2.

For obvious reasons I do not find this to be an acceptable answer!
What do you think?  What am I missing?  Is there any meaningful
interpretation to "degrees of freedom" in a general theory of
inference?


T. Scott Thompson               thompson@atlas.socsci.umn.edu
Department of Economics
University of Minnesota
From: jerry@ginger.hnrc.tufts.edu (Jerry Dallal)
Newsgroups: sci.math.stat
Subject: Re: Fwd: Standard Deviation.
Date: 15 Aug 92 22:29:07 GMT
Organization: USDA HNRC at Tufts University

In article <seX2yRq00Uh785H2EB@andrew.cmu.edu>, dr3u+@andrew.cmu.edu (Daniel Read) writes:
> 
> Can someone explain why calculating the Standard Deviation (SD),
> for small samples, with (n-1) in the denominator is better than
> doing so with (n) in the denominator?  


I'm totally confused by some of the answers to this question.
Using n-1 in the denominator gives an unbiased estimate of the variance.

E (sum[(xi-xbar)**2]) = E (sum[(xi-mu)**2]) - E(sum[(mu-xbar)**2])
                      = E (sum[(xi-mu)**2]) - nE[(mu-xbar)**2]
                      = n sigma**2          - n sigma**2 / n
                      = (n-1) sigma**2

Of course, there's nothing magic about unbaisedness.  You might prefer a
maximum likelihood estimate (n in the denominator) or a minimum mse estimator
(n+1 in the denominator).  The reason n-1 is used is that you have to use
*something* and n-1 makes the cleanest formulas.

For example, Lindgren and Friedman, Pisani, and Purves define their sd with an
n in the denominator.  This means that their formula for CIs involving the t
distribution becomes

                  xbar +- c s / sqrt(n-1)
instead of the usual
                  xbar +- c s / sqrt(n)

(Note:  All of this assumes a whole bunch of qualifications and conditions that
I haven't stated explicitly, beginning with the fact that the variance should
exist!)
Newsgroups: sci.math.stat
From: victorw@ee.ubc.ca (Victor J. K. Wong)
Subject: simulation of 2-dimensional Poisson distribution
Organization: University of BC, Electrical Engineering
Date: Sun, 16 Aug 1992 23:48:29 GMT


Hi reader,

For a 2-dimensional homogenous Poisson distribution, 
p(i,A) = exp(-A*d)*(A*d)^i/i! where p(i,A) is the probability
of i occurrences in an area A, d is the spatial density
(number of occurrences per unit area). Similar to the
one-dimensional Poisson distribution, 2-dimensional one
also has the memoryless property. 

To simulate a 2-dimensional Poisson environemnt, one can
divide an area A into many small areas with equal size B.
If B is small enough, p(0,B) + p(1,B) is very close to 1
and hence, p(i,B), for i > 1, can be discarded. For each
small area B, the probability of occurrence is p(1,B) and
the probability of no occurrence is p(0,B), and this is
a binominal process. If B is very small, one can assume
that the location of the occurrence is the center of the 
area B If this process repeats for each area B, the 
distribution of of any area in A is a 2-dimensional
Poisson. However, such a simulation takes long time, does
anyone has better suggestion for a faster simulation of
a 2-dimensional Poisson environment? If you have any
suggestion, please email to me (email address victorw@ee.ubc.ca).
Your help will be very much appreciated.

Victor
From: wvenable@algona.stats.adelaide.edu.au (Bill Venables)
Newsgroups: sci.math.stat
Subject: Re: |STAT now available via anonymous ftp
Date: 15 Aug 92 04:58:16 GMT
Organization: Department of Statistics, University of Adelaide
In-reply-to: perlman@cis.ohio-state.edu's message of 13 Aug 92 06:12:50 GMT

>>>>> "Perlman" == Gary Perlman <perlman@cis.ohio-state.edu> writes:

Perlman> To unpack the archive, you need uudecode, crypt, uncompress, and tar.
Perlman> To ensure that the distribution restrictions on |STAT are maintained,
Perlman> a password has been attached to the archive.
Perlman> To obtain the password from me, you must send me a message with:
Perlman> 	I AGREE TO ADHERE TO THE CONDITIONS OF USING |STAT.
Perlman> 	I AGREE THAT I WILL NOT SHARE THE |STAT PASSWORD WITH OTHERS.
Perlman> After obtaining the file, which you can also request from me via email,
Perlman> you should run the following commands:
Perlman> 	uudecode < stat.tar.Z.crypt.uu
Perlman> 	crypt < stat.tar.Z.crypt > stat.tar.Z
Perlman> 	uncompress stat.tar.Z
Perlman> 	tar xvf stat.tar

Just a couple of neutral comments.

uuencoding is not necessary for files transferred by remote ftp, so long as
"binary" mode is set during transmission.

I suppose it is well known that the "crypt" facility is still an illegal
export from the US and (in theory) should not exist on any machine outside
the US.  So persons like me outside the US intending to get |STAT this way
will need first to get an account somewhere inside the US, remote login to
it, ftp it there first, decrypt it there, re-ftp it home and so on.  But if
that's the deal, then I guess that's the deal.

The last two lines above can be done as one without explicit uncompressing:

	zcat stat.tar.Z | tar xvf -

You can sometimes avoid discs overflowing and write failures this way.

wnv.
--
___________________________________________________________________________
Bill Venables, Dept. of Statistics, | Email: venables@stats.adelaide.edu.au
Univ. of Adelaide, South Australia. | Tel: +61 8 228 5412  Fax: ...232 5670
From: klin@ucrengr.ucr.edu (kenneth m. lin)
Newsgroups: sci.math.stat
Subject: Looking for Good Referral Services
Date: 16 Aug 92 20:28:16 GMT
Distribution: usa
Organization: University of California, Riverside


Does anybody know a good job referral service for entry-level statistican? 
I have contacted some headhunters advertised in AmStat News and other magazines
but they were not very helpful, perhaps because I am just a recent grad with 
very little experiences and because I only has a M.S. degree.  Not graduating 
from famous school isn't helping me with my job serach either.


Thanks in Advance,

Kenneth M. Lin
Department of University, Riverside

email address: klin@ucrengr.ucr.edu

Newsgroups: sci.math.stat
From: baer@uwovax.uwo.ca
Subject: LIMDEP (info. request)
Organization: University of Western Ont, London
Date: Thu, 13 Aug 1992 12:42:00 GMT

Does anyone on sci.math.stat know the e-mail address of 
William Greene, the author of LIMDEP? 
---------------------------------------------------------------------
Douglas Baer, Department of Sociology,
University of Western Ontario, London, Canada N6A 5C2
Internet: BAER@UWOVAX.UWO.CA   

Newsgroups: sci.math.stat
From: thompson@atlas.socsci.umn.edu (T. Scott Thompson)
Subject: Re: Degrees of Freedom Was: Re: Standard Deviation.
Reply-To: thompson@atlas.socsci.umn.edu
Organization: Economics Department, University of Minnesota
Date: Sat, 22 Aug 1992 17:54:01 GMT

Andrew.Robinson@anu.edu.au writes:

>In article <thompson.714338397@kiyotaki.econ.umn.edu>
>thompson@atlas.socsci.umn.edu (T. Scott Thompson) writes:
>>hougen@uirvlh.csl.uiuc.edu (Darrell Roy Hougen) writes:
>>
>[deletia]
>>
>>qualms, consider the following.  Suppose my data consist of a single
>>number, say X.  Suppose that I am given two pieces of information
>>about X (but not X itself):
>>
>>           (1)  I can observe Y := X**2
>>           (2)  I can observe Z := {  1 if X > 0
>>                                   { -1    otherwise
>>
>>How many "pieces of information" do I now have?  Clearly I can recover

>I'm going to go _way_ out on a limb here and say that I think that you have
>_one_ piece of infomation.  If you don't have X then your data doesn't consist
>of a single number, it consists of a single observation from which you have
>measured a two variable vector - [Y,Z].  You know  that there is an X and you
>want to use Y and Z to estimate it using a predetermined model, which you have
>specified.  This seems to me to be a non-straightforward case of a multivariate
>regression, with two dependent variables and one observation.

>Comments?

Well first, I don't have to estimate X since I can calculate it
exactly from Y and Z.  Secondly, I agree that the single data point X
contains the exact same information as the vector (Y,Z).  I will even
agree (although I am not sure that this is so obvious) that the number
of "pieces of information" in X (hence in (Y,Z) is one.  After all,
the latter could simply be a normalization of units.

The problem is to relate this to the usual degrees of freedom
accounting which seems to require that independent "pieces of
information" add up in some way.  For example, we seem to think of Y
and Z as independent pieces of information because knowledge of one is
totally uninformative about the other.  According to the adding up
rule, then, (Y,Z) should be 2 pieces of information, not one.  Clearly
we agree that this is not so.  The question is "why not?"

I think that the definition of "independent pieces of information"
that is implicitly used in the informal interpretation of "degrees of
freedom" does not satisfy an adding up rule, and thus fails to provide
a good interpretation for what "degrees of freedom" really represent.
Assuming they represent anything.  I am skeptical about this.
-- 
T. Scott Thompson              email:  thompson@atlas.socsci.umn.edu
Department of Economics        phone:  (612) 625-0119
University of Minnesota        fax:    (612) 624-0209
--
T. Scott Thompson              email:  thompson@atlas.socsci.umn.edu
Department of Economics        phone:  (612) 625-0119
University of Minnesota        fax:    (612) 624-0209
From: wlk@aber.ac.uk (Warren L Kovach)
Newsgroups: sci.math.stat
Subject: Re: Powell's Min. w/ Brent's mod. wanted
Date: 13 Aug 92 07:35:57 GMT
Organization: University of Wales, Aberystwyth

In article <11545.2a890722@cpva.saic.com> inlowm@cpva.saic.com writes:
>See "Numerical Recipes" by Press (? -- three other authors).
>There is a fortran version and a c version.  If you use the
>c version be careful -- I have a friend who claims there are
>some bugs in it.

Numerical Recipes is also available in a Pascal version, in case that
is your favorite flavor!

-- 
Dr Warren L. Kovach			           Internet: WLK@ABER.AC.UK 
Institute of Earth Studies						    
University of Wales, Aberystwyth		     tel. +44 (0970) 622626 
Aberystwyth, Wales SY23 3DB U.K.