HDTenEightyP/NetNews huggingface dataset:
From: steve@cogsci.ed.ac.uk (Steve Finch)
Newsgroups: sci.math.stat
Subject: Re: Let's Make a Deal!
Date: 28 Sep 92 19:00:32 GMT
Organization: Centre for Cognitive Science, Edinburgh, UK

packer@amarna.gsfc.nasa.gov (Charles Packer) writes:

>Wait a minute, I thought I understood this a few months
>ago when it was discussed here. But now I write down what
>I understand to be the sample space and it says: don't
>bother to switch, it's a fair game:

>Assume for simplification that the prize is always behind door A.
>Probabiliy of each outcome is listed:
>I choose A, host chooses B, I stay with A 1/12 win
>I choose A, host chooses B, I switch to C 1/12
>I choose A, host chooses C, I stay with A 1/12 win
>I choose A, host chooses C, I switch to B 1/12
>I choose B, host chooses C, I stay with B 1/6
>I choose B, host chooses C, I switch to A 1/6  win
>I choose C, host chooses B, I stay with C 1/6
>I choose C, host chooses B, I switch to A 1.6  win

>Add the "win" probabilities together, and they come out
>to .5. I must have forgotten something...

Your analysis of the problem is entirely correct, and indeed
illuminating, and shows what happens if you toss a coin to decide your
strategy.  

Your interpretation of the solution to the problem is incorrect.  The
reason it pays to switch is because the host's action is forced if you
pick an incorrect door, so the host's action gives you information
about where the prize actually is which you can exploit in determining
your strategy.  Adding the probabilities of winning together above is
an answer to the problem `what if I tossed a fair coin to decide my
strategy?' (ie.  ignore the host's information in determining my
strategy).  The correct interpretation of P(win|switch) is to delete
all the "stay" lines above, and multiply the remaining probabilities
by 2 (since P(switch) has just increased by a factor of 2), and then
it becomes clear that

P(win|switch)=1/3 + 1/3,

these components coming from the situations where the host's action
was forced (ie. informative).  This is not surprising, since a-priori
there is a 2/3 chance that the host's action will tell you where the
prize is (ie. if you initially pick a loser).

Steve.
------------------------------------------------------------------------------
When you steal from one person, it's called Plagiarism-
When you steal from many, it's research.                       - Wilson Misner
------------------------------------------------------------------------------
Steven Finch   Phone: +44 31 650 4435           | University of Edinburgh
					        | Centre for Cognitive Science
ARPA:  S.Finch%ed.ac.uk@nsfnet-relay.ac.uk      | 2 Buccleuch Place
JANET: steve@uk.ac.ed.cogsci                    | Edinburgh            EH8 9LW
From: robt@agsm.unsw.edu.au (Rob Trevor)
Newsgroups: sci.math.stat
Subject: Re: garch modelling software
Date: 28 Sep 92 01:14:55 GMT
Followup-To: sci.math.stat
Organization: Australian Graduate School of Management

In article <1992Sep23.205017.25101@fnbc.com>, jja@fnbc.com (Joseph Alotta)
wrote:
> 
> 
> Does anyone know a software package or a routine that can do GARCH  
> modelling?
> 
> Can someone also recommend some papers I might read?
> 

Well, you just need a nonlinear maximisation routine.  I do lots of GARCH
stuff (including multivariate versions) using the RATS (regression and time
series) package which is an extremely powerful, yet not too expensive,
econometrics package with an inbuilt high level programming language.  (I
run it on workstations and Macs, but PC and mainframe versions are also
available.)  But you could use anything that will allow you to maximise a
user supplied objective function.

If you want to start out on Generalised AutoRegresssive Conditional
Heteroescadiscity models, then you must read the survey in the Journal of
Econometrics about a year ago.  (Sorry, don't have the exact ref at hand at
the moment.  The whole issue of this journal consisted of ARCH models in
finance.  EMail me if you can't find it.)  This considerably updates the
survey published in Econometric Reviews in 1986.

Rob

------------------------------------------------------------------------------

Rob Trevor  Associate Professor, Australian Graduate School of Management
            University of New South Wales
            PO Box 1, Kensington, NSW, AUSTRALIA    2033

Internet:  robt@agsm.unsw.edu.au  or  R.Trevor@unsw.edu.au

VOICE: +61 (2) 931-9274    FAX: +61 (2) 662-7621  or   +61 (2) 662-2451

------------------------------------------------------------------------------
Newsgroups: sci.math.stat
From: bill@bessel.as.utexas.edu (William H. Jefferys)
Newsgroups: sci.math.stat
Subject: Re: Let's Make a Deal!
Date: 29 Sep 92 04:14:29 GMT
Organization: McDonald Observatory, University of Texas @ Austin

In article <80686@ut-emx.uucp> I wrote:

#I think there is more than mathematicians'
#and statisticians' egos involved. MvS's ego is
#equally involved, and her response to a critical
#article in (what?) Journal of Mathematical
#Education (?)--the reference was posted recently--
#shows this very clearly.

Mike Minnotte posted the correct reference:

. The November, 1991 issue of The American Statistician had an article
. by Morgan, Chaganty, Dahiya, and Doviak which analyzes the problem
. under fairly general assumptions.  It turns out that _if_ the host
. always shows a goat, you always do better to switch, but you only
. do as well as 2/3 if ,given that you have chosen the door with the
. car to begin with, the probabilities are equal that he chooses each
. of the other two doors.
. 
. The issue also contains a fairly hostile letter in response by
. vos Savant.  If you're interested in the problem, it's worth
. checking out.

Bill
Newsgroups: sci.math.stat
Subject: E-mail address of ASA?
From: bmann@desire.wright.edu
Date: 25 Sep 92 16:14:40 EST
Organization: Wright State University 

I seem to recall a fairly recent announcement that it is now possible to
send e-mail to the national ASA office, but I have lost the address and
couldn't find it in recent AmStat News.  Does anyone have it handy?  
Our computer tends to lose postings rather quickly, so I would appreciate
direct e-mail: bmann@desire.wright.edu.  Thanks.

Barbara Mann, Dept of Math and Stat, Wright State University, Dayton, OH
bmann@desire.wright.edu
Newsgroups: sci.math.stat,sci.math.num-analysis
From: charlie@umnstat.stat.umn.edu (Charles Geyer)
Subject: Re: Gibbs sampling and annealing (was: Help!  Book recommendations?)
Organization: School of Statistics, University of Minnesota
Date: Mon, 5 Oct 1992 22:50:03 GMT

In article <92Oct5.123636edt.472@neuron.ai.toronto.edu> radford@cs.toronto.edu
(Radford Neal) writes:

> The technique consists (in its simplest form) of splitting the
> simulation up into several phases, each associated with a certain
> "temperature", T. These temperatures decline, starting at a high
> value, and ending at T=1. In each phase, a Markov chain that has an
> equilibrium distribution proportional to P(x)^(1/T) is simulated,
> where P(x) is the distribution we wish to sample from. The start state
> for each phase is the final state from the previous phase.
>
> Once a temperature of one is reached, or perhaps a bit later, one
> starts using the states visited by this final chain (with T=1) to
> calculate expectations with respect to the distribution given by
> P(x). 

But this doesn't help much.  All it attempts to do is produce one
realization that is "almost" from the stationary distribution to
use as the start of the run.

But if the chain mixes very slowly, the average will be near the starting
point, but not near the expectation one wants to calculate.

On the other hand, if the chain mixes rapidly, the "annealing" isn't
needed.

A rather different algorithm, in which the chain is slowly "annealed"
and its final state is treated as a single realization from the stationary
distribution (and one needs a separate annealing run for each realization
simulated) makes some sense, but is horribly slow and (as I said) throws
Markov chain theory out the window.

-- 
Charles Geyer
School of Statistics
University of Minnesota
charlie@umnstat.stat.umn.edu
Newsgroups: sci.math.stat
From: zrajs01@sunap1.zdv.uni-tuebingen.de (Jochen Schmid)
Subject: Testing random number generators
Organization: Comp. Center (ZDV) U of Tuebingen, FRG
Date: Tue, 6 Oct 1992 15:17:21 GMT

The results of the statistical tests for goodness of randomnumber generators
proposed by D.E. Knuth :Semi-numerical Algorithms, depend on the
choice of the generators seed. What seed should be used?


       Jochen Schmid, Abteilung Anwendungssoftware, ZDV
       Universitaet Tuebingen
       Tel.: + 07071-29 6018, Fax : + 07071-29 5912 
       Internet mail: schmid@mailserv.zdv.uni-tuebingen.de


--
       Jochen Schmid, Abteilung Anwendungssoftware, ZDV
       Universitaet Tuebingen
       Tel.: + 07071-29 6018, Fax : + 07071-29 5912 
       Internet mail: schmid@mailserv.zdv.uni-tuebingen.de
From: mtd@uctgeo.geo.uct.ac.za (Marian Tredoux)
Newsgroups: sci.math.stat
Subject: LISREL software request
Summary: Looking for software to test for equivalent of covariant matrices
Keywords: Equivalence of covariant matrices, Genrich test
Date: 2 Oct 92 13:09:59 GMT
Article-I.D.: uctgeo.mtd.718031399
Organization: University of Cape Town

We are trying to locate a copy of the LISREL software that checks for
equivalence between covariant matrices, or any other software that runs
data though the Genrich test.

Please reply to my personal email address, as I do not follow this group
regularly.

Many thanks in advance, 
Marian Tredoux.
From: kt14+@andrew.cmu.edu (Kok Hooi Tan)
Newsgroups: sci.math.stat
Subject: DIEHARD test of randomness
Date: 30 Sep 92 09:26:00 GMT
Article-I.D.: andrew.4emSAcS00j618xs0YB
Organization: Doctoral student, Statistics, Carnegie Mellon, Pittsburgh, PA


 Hello,
 Are the codes of the so called DIEHARD tests of random numbers by Marsaglia 
 publicly available, say in some ftp site ?
 Thanks,

---------------------------
Kok Hooi Tan                    
Dept of Statistics              
Carnegie Mellon University      
Pittsburgh, PA 15213            
---------------------------
From: esran@warwick.ac.uk (Mr P H Chan)
Newsgroups: sci.math.stat
Subject: Need help
Date: 27 Sep 92 14:55:39 GMT
Organization: Computing Services, Warwick University, UK

Hi! I am a new reader of the group and need help from the expert in this
group. I am a post-graduate engineering student and working on pattern
recognition. The aim of my project is to assign a vector, V, to one of the
classes. My first task is to reduce the dimension of vector. My question is

1)  How do I rank/order the element of vector according to their
    discriminating information? In other words, is there any method
    to measure the discriminating ability of esch member of vector?
2)  How can I determine the minimum dimension of reduced vector?

Any suggestion and reference is welcome.

Thank you in advance.


P. H. Chan

-- 
------------
Department of Engineering	|	E-mail :
University of Warwick		|	es283@uk.ac.warwick.eng
Coventry U.K.  CV4 7AL		|	esran@warwick.ac.uk
Xref: sparky sci.math.stat:2066 sci.math.num-analysis:2926
Newsgroups: sci.math.stat,sci.math.num-analysis
From: radford@cs.toronto.edu (Radford Neal)
Subject: Re: Gibbs sampling and annealing (was: Help!  Book recommendations?)
Organization: Department of Computer Science, University of Toronto
Date: 5 Oct 92 16:37:05 GMT

In article <1992Oct4.192217.27919@news2.cis.umn.edu> charlie@umnstat.stat.umn.edu (Charles Geyer) writes:

>> ... Simulated annealing
>> is a technique that can be used in conjunction with Gibbs sampling 
>> (or other Markov chain methods, such as the Metropolis algorithm) 
>> in order to reach the final equilibrium distribution more quickly.
>
>Depends on how you think about Markov chain Monte Carlo.  The
>Metropolis-Hastings algorithm produces a Markov chain with a known
>stationary distribution.  This "annealing trick" doesn't.  So far
>from mixing MCMC and simulated annealing, it is not MCMC at all.

Perhaps I wasn't clear. 

The technique consists (in its simplest form) of splitting the
simulation up into several phases, each associated with a certain
"temperature", T. These temperatures decline, starting at a high
value, and ending at T=1. In each phase, a Markov chain that has an
equilibrium distribution proportional to P(x)^(1/T) is simulated,
where P(x) is the distribution we wish to sample from. The start state
for each phase is the final state from the previous phase.

Once a temperature of one is reached, or perhaps a bit later, one
starts using the states visited by this final chain (with T=1) to
calculate expectations with respect to the distribution given by
P(x). 

This procedure has as much, and as little, formal justification as
simply running a chain that has P(x) as its invariant distribution for
the same number of total steps before using the states to compute
expectations.  


>Are there any practically useful convergence results for such schemes
>once Markov chain theory is thrown out the window?

One doesn't throw Markov chain theory out the window, one simply
recognizes its limiations.

Generally, the easily obtainable bounds on convergence of the Markov
chain methods when applied to interesting problems do not justify
their practical use. It is indeed nice to know that convergence is
guaranteed asymptotically, but the warm feelings this gives are
equally available to those using annealing, since the annealing
phases, where T>1, occupy only a finite initial portion of the chain.
In practice, one is relying on intuition and empirical experience with
similar problems in the past, along with the knowledge that there
aren't any better techniques, so what else is one to do?

Empirically, the annealing procedure is often found to give better
convergence than just starting at T=1. There are no generally
applicable formal results. It is easy to construct problems for which
annealing makes things worse, and for which, in fact, it is better to
start at a temperature _less_ than one and gradually warm up to T=1.
However, deciding to do the whole simulation at T=1 is just as much
an "act of faith" as is using some annealing schedule.

    Radford Neal
Newsgroups: sci.math.stat
From: bill@alembic.acs.com (Bill Hatch)
Subject: gappy/noisy time series
Organization: Colman Research
Date: Thu, 1 Oct 1992 23:56:59 GMT

Newsgroups: sci.stat.math
Subject: gappy/noisy time series
Followup-To: Message-ID: <1992Sep29.234100.7104@netnews.whoi.edu>
Distribution: world
Organization: Colman Research


Rich Pawlowicz of Organization: Woods Hole Oceanographic Institution
asks the following:

>I am trying to generate spectral estimates from time series which have
>2 problems:
>
>   1) dropouts, i.e. gaps in the time series (perhaps 20% of the values)
>   2) error spikes (perhaps 2% of the values)
>
>The time series is a geophysical one (i.e. reddish spectrum), with
>perhaps a few small peaks (tides and what-not).
>
>The standard window-fft-and-average techniques seem to be inadequate
>for this - I can't think of a good way to handle the gaps (other than
>ignoring them by removing them from the series), and the spikes tend to
>give a lot of "white" energy.
>
>Median filtering can get rid of some of the spikiness, but it does bad
>things to some of the other parts of the signal.
>

My suggestion is to try the following:

1. take a dft (direct Fourier transform) of the unequally spaced
series. Take the inverse dft at equally spaced intervals.  The real
part will be your recovered "signal" and the imaginary part is
the noise estimate or residuals. I seem to remember that this is
equilivant to doing a least squares estimate.  

2. plot the power spectrum and identify a reasonable cutoff frequency
for a low pass filter to remove the effect of the spikes.  Alternatly,
you could just taper off the high freq. terms before taking the
inverse dft above.

3. let me know how you make out.  Also, if possible, i would like to
get a copy of your ``problem'' time series.  I have put together a
time series package in xlisp (submitted to the statlib archieve) and
i would like to exercise it on some real problems.

bill hatch
-- 
Bill Hatch  (301)470-3839(w)  (301)441-1675(h)
Coleman Research, 14504 Greenview Drive, Laurel MD 20708
All opinions expressed are my own - dont blame my employer or alembic systems
From: SXC101@psuvm.psu.edu
Newsgroups: sci.math.stat
Subject: Q: distances between distributions
Date: 6 Oct 92 21:30:17 GMT
Organization: Penn State University

Hi:

1) there are N points in the (x-y) space. Each point has
   has a distribution around it - call it error distribution
this distribution has an elliptic shape. its area is proportional to
   (1+abs(x)+abs(y))**2.

2) is there a measure to tell me something about separation between
   the distribution?

3) say instead of mapping (x,y) directly, we map (logx, logy).
this definitely reduces the error distribution but at the same
   time compress the distances between points. The measure
devised for 2) should be able to reflect this.


greatly appreciate any help.

chakra
Newsgroups: sci.math.stat
From: behzad@ec.ecn.purdue.edu (Behzad M Shahshahani)
Subject: expected value of inverse sample covariance?
Organization: Purdue University Engineering Computer Network
Date: Thu, 1 Oct 1992 20:19:06 GMT


I have been trying to find the expected value of inverse of S, the
unbiased covariance estimte. Does anyone know what this expected value is, 
or any references??
                                                         T
E{inverse(S)} = ??, where S=1/n-1 (sum((xi-xbar)(xi-xbar) )

Thanks in advance.

-Behzad Shahshahani
Newsgroups: sci.math.stat
From: rizzo@cbnewsf.cb.att.com (anthony.r.rizzo)
Subject: Re: Ross Perot and Statistics
Organization: AT&T
Date: Fri, 2 Oct 1992 17:18:05 GMT

In article <1992Oct2.111839.7278@iscsvax.uni.edu> howard@iscsvax.uni.edu writes:
>Was it just me, or did anyone else cringe a bit at Ross Perot's
>methods of "asking the volunteers" whether he should run for 
>President?
>
>Some problems I saw with his "technique" were:

................... much stuff deleted ...............

This, clearly, is one case where sample statistics are
irrelevant.  We'll test the ENTIRE voting population in November.
Lets hope that the correct techniques will be used THEN.

Tony Rizzo
From: behzad@ec.ecn.purdue.edu (Behzad M Shahshahani)
Newsgroups: sci.math.stat
Subject: Re: expected value of inverse sample covariance?
Date: 2 Oct 92 14:06:40 GMT
Organization: Purdue University Engineering Computer Network


-behzad
Newsgroups: sci.math.stat
From: thompson@atlas.socsci.umn.edu (T. Scott Thompson)
Subject: UMVU estimation for infinite-dimension models
Summary: Does the Rao-Blackwell theorem generalize to nonparametric models?
Keywords: nonparametric model, unbiased estimation, UMVU
Reply-To: thompson@atlas.socsci.umn.edu
Organization: Economics Department, University of Minnesota
Date: Tue, 29 Sep 1992 19:15:34 GMT

Perhaps one of the theorists out there can address this question:

The literature on estimation given i.i.d. sampling and a parametric
probability models includes three important results:

(1) The Cramer-Rao lower bound on the asymptotic variance of any
"regular" estimator.

(2) The Cramer-Rao lower bound on the variance of a uniformly unbiased
estimator.

(3) If T is a complete sufficient statistic (with respect to a given
probability model) and b is a uniformly unbiased estimator of a given
parameter of interest then there is an essentially unique minimum
variance unbiased estimator given by E(b|T).

I am interested in whether/how these generalize to nonparametric
probability models.  For example, the literature on semiparametric
efficiency bounds clearly provides a generalization of (1) to
semiparametric and nonparametric probability models.  I am not aware
of any nonparametric forms of the finite sample results (2) or (3),
however.

Do any of you know of any pointers/results on this?

The asymptotic generalization of (1) to nonparametric models is
roughly equivalent to looking for a worst case parametric submodel.
It occurs to me that a similar approach might yield generalizations of
(2) and (3).

To give a concrete example of the kind of questions that I have in
mind, suppose that I have an i.i.d. sample and an estimator of the
U-statistic form that is an unbiased estimator of some parameter of
interest.  Suppose that the true distribution from which the data were
sampled is known to lie in some set of probabilities, possibly
restricted in some qualitative fashion, but not so much as to admit a
finite-dimensional parameterization.  How might I check whether or not
my estimator is UMVU?

I will summarize replies.  Thank you in advance.
--
T. Scott Thompson              email:  thompson@atlas.socsci.umn.edu
Department of Economics        phone:  (612) 625-0119
University of Minnesota        fax:    (612) 624-0209
From: Decker@Themis.CS.UMASS.EDU (Keith Decker)
Newsgroups: sci.math.stat
Subject: Answer: Binomial Order Statistics
Date: 2 Oct 92 14:55:11 GMT
Reply-To: decker@ariel.cs.umass.edu
Organization: Computer and Information Science, University of Massachusetts, Amherst, MA


Thanks to everyone who responded to my question about the
density of the max order statistic for binomially distributed
random variables, including:

B. Narasimhan (naras@cda.mrs.umn.edu)
Herman Rubin  (hrubin@pop.stat.purdue.edu)
Tom Permutt   (tpermutt@medinfo.ab.umd.edu)
Terry Moore   (T.Moore@massey.ac.nz)
Fouad Younan  (sasfgy@unx.sas.com)
Ernie Scheuer (VCMGT001@vax.csun.edu)

By the third message, I had enough info to work it out, and the
final message from Ernie Scheuer confirmed my answer.
The last treatment was the most complete, so I will include it
here for anyone who also wanted to know the answer.

Thanks again!
/Keith

------------------------------------------------
From: VCMGT001@vax.csun.edu (Ernie Scheuer)
Subject: Dist. of the max of several binomials

Here's a treatment of the query you posted in sci.math.stat.

Y1,...,YK are iid binom(n,p).

Let:
S(y;n,p) = sum(j=0 to j=y)[(n-binomial-j)*(p^j)*((1-p)^(n-j))]

That is, S(y;n,p) = P[Yi <= y], i = 1,..., K .

Let Y = max(Y1,...,YK) .  Then
                              
P[Y <= y] = P[max(Y1,...,YK) <= y] = P[Y1 <= y,...,YK <= y]
          
          = {P[Yi <= y]}^K = {S(y;n,p)}^K


You wanted the density for Y:

P[Y = y] = P[Y <= y] - P[Y <= y-1] for 0 < y <= n;
P[Y = 0] = (1-p)^(nK).

For y > 0 the expression for P[Y = y] is the difference of two
polynomials of degree K*n;  if you want this closed form, I 
suppose you could get it, but it wouldn't be "pretty."


There are tables of, and computer programs for, S(y;n,p).  


Caution:  S(y;n,p) could properly be called the cumulative
          distribution function for the binom(n,p) random
          variable.  The term cumulative binomial distribution
          is sometimes used for 1 - S(y-1;n,p), the probability
          that a binom(n,p) random variable is greater than or
          equal to y.  Moral: look carefully at binomial tables 
          or programs to see exactly what they provide.

[...]