tttjjj/elt_bench huggingface dataset:
one that works for the paper is [here](http://ba.stat.cmu.edu/journal/2009/vol04/issue04/fuquene.pdf).  It was on a conference "objective bayes 09" website (the Valencia meetings).  I don't think the slides will be available anymore, as the website for the conference has been taken down... :(  pity, it was a good set of slides.  That horshoe prior does look interesting in the link you provided.
Could you also please check *definition two*? $\\sum_{i=0}^{t-1} \\epsilon_{t-1}$ does not look right: what happens to $i$ in the expression?
@AlexKreimer: To reduce biasing by extraordinary big/small elements - it was shown that for this specific kernel it increased performance.
@whuber Yes, that's true. In this case we could use multivariate Gaussian noise, for example. But as I know Excel regression build it based on just errors for $Y$, so this would not be correct. Additionally, as for prediction we are given an exact point $X_{new}$ we could project this multivariate gaussian into one dimension $Y$. And that would give as a solution like in the answer.
Remember that the very purpose of the bootstrap is to give a non-parametric estimate of the sampling distribution, so bi-modality is not necessarily a problem. If we had good reasons to expect that the real sampling distribution is bimodal, then the result you found would be just fine. In practice it often means you have outliers, which might or might not be a problem depending on the exact circumstances. The solution would be to know your data and know what you want to do with it. Ignoring parts of the estimated sampling distribution as you seem to propose is unlikely to be the answer.
If you rewrite the original model as $Y \\sim b \\frac{X_0}{X_1 X_2}$ where $b = \\frac{a_0}{a_1 a_2}$ then you can identify $b$  -- e.g. by the kind of log-transform you suggest, or through a nonlinear least squares model, or through a GLM, or a number of other ways - but you cannot identify the individual $a's$ in your formula. (Is this worth working into an answer? Do you have followup issues?)
+1, Unbelievable that this was made available in SPSS!
If you think there are too many zeros, you may want to run a zero-inflated Poisson (zip) model, instead of negative binomial, which is more for overdispersion.
I would consider adding more tags to your question (at the very least, I would add the `time-series` tag).
Probability of what?
It is an intercept only model, where the only parameter is related to the proportion of '1' in the population.
If you [Google](https://www.google.com/search?q=probit+in+r) it, you will find some nice tutorials like [this](http://www.ats.ucla.edu/stat/r/dae/probit.htm).
This question is unclear because rotation of factors or components, even orthogonal one, has nothing to do with "eigenvalues". Some factor analysis methods do not deal with eigenvalues even prior rotation. Perhaps you meant the term "factor variances" saying "eigenvalues"?
The Fundamental Theorem of Calculus says that areas under the first derivative of a function are given by differences of values of that function itself, so there's not much more to say. What is the statistical motivation behind this question? What is your calculation attempting to represent?
For the MARS, it may work. But, I am not sure how I can convert the obtained equation to a correlation-like score.
@suncoolsu - note that a "beta-bernoulli" distribution is just another bernoulli distribution but replacing $p_i\\to\\frac{\\alpha}{\\alpha+\\beta}$.  This is becase ${1\\choose x_i}\\frac{B(\\alpha+x_i,\\beta+1-x_i)}{B(\\alpha,\\beta)}=\\frac{\\alpha^{x_i}\\beta^{1-x_i}}{\\alpha+\\beta}$.  So basically by mixing over $p_i$ you are applying the binomial approximation here $p_1=p_2=\\dots=p_n$.
@whuber, thanks for the comment. I have added some clarifications to the question. please let me know if any other question.
Isn't this essentially the [same question](http://stats.stackexchange.com/questions/61626/i-log-transformed-my-dependent-variable-can-i-use-glm-normal-distribution-with) you asked earlier, apart from the mention of mixed models?
A common reason for such differences is the fact that likelihood is only [defined up to a multiplicative constant](http://en.wikipedia.org/wiki/Likelihood_function): "*More precisely, then, a likelihood function is any representative from an equivalence class of functions,
$\\mathcal{L} \\in \\left\\lbrace \\alpha \\; P_\\theta: \\alpha > 0 \\right\\rbrace, \\,$ 
where the constant of proportionality $α > 0$ is not permitted to depend upon $θ$, and is required to be the same for all likelihood functions used in any one comparison.*" Log-likelihood may in turn be shifted by an arbitrary constant. ...(ctd)
(+1) Nice answer!
"*if you wanted to approximate a binomial distributed variable with a two-component mixture normal*" - optimizing what criterion? The answer to that could certainly impact the answer to "*is there a closed form for the answer*". As for 'preferred approaches', my preferred approach to answering such question is to first try to find out what the underlying problem is, and what aspects are more critical - whether a closed form is needed more than a good fit in the tail, for example.
Motulsky: I am comparing hazards ratios for 5 groups (with 1 of the groups as the reference group) based on some value of a continuous variable.
Is it 0.01% or 0.10%? 30 out of 30,000 is 0.10%.
@ abi: Depending on sample size, Spearman's and Kendall's coefficients are relatively similar in terms of power and MSE to Pearson's with normally distributed data, and are far superior with even slight data contamination.
Your workaround appears to be Wald-based confidence intervals, not prediction intervals. See [mail 1](http://r.789695.n4.nabble.com/Re-Can-not-get-a-prediction-interval-from-Predict-td885947.html) and [mail 2](https://stat.ethz.ch/pipermail/r-help/2003-May/033165.html) on the R-help list.
@OweJessen That's normally called a compound distribution (in this case a [compound Poisson](http://en.wikipedia.org/wiki/Compound_Poisson_distribution)) rather than a mixture; if jbowman has understood you correctly you should probably fix your question.
I agree with this advice, but it looks like neither your code nor the formula appear to generate a *truncated* exponential.
0.05 was a typo yes, I am using 0.005.
But why do you say I am throwing out every event after the first?
Unfortunately, I think you're right about jags. I get a "Unable to find appropriate sampler" error when I put a prior on Phi1 that's an inverse() applied to a dwish() r.v.
@Glen_b Latex shows in comments just fine (the "Show math as TeX Commands" option), and can be copy-pasted.
@user1665355 I presumed you understood how regression or classification trees were built? The trees in RF are no different (except perhaps in stopping rules). Each tree splits the training data into groups of samples with similar "values" for the response. The variable and split location (e.g. pH > 4.5) that best predicts (i.e. minimises "error") forms the first split or rule in the tree. Each branch of this split is then considered in turn and new splits/rules are identified that minimise the "error" of the tree. This is the binary recursive partitioning algorithm. The splits are the rules.
I'm sorry but I fail to see how this answers the question. I did not ask for a robust prior, but rather for a robust *model*.
What you are describing is very close to [principal components analysis](http://en.wikipedia.org/wiki/Principal_component_analysis).
Can you explain further what you mean? It is very common to test whether the slope parameter for a variable is different from zero. I would call that "hypothesis testing". Are you unaware of that, or do you mean something different? What constitutes a scenario for your purposes?
"data" refers to a collection of variables and observations in the context of a study. You mean to say "two types of variables". Variables with missing values are obviously observ*able* because they were observ*ed*. I don't understand your comment, but I suggest providing an example to motivate the question.
Thanks but I had read that before. Even if we ignore the obvious issues with the "auto" part for now, there are issues with the "arima" part, especially when it is extended to include seasonal models. Seasonal ARIMA models have been strongly criticized by P.J. Harrison, C Chatfield and some other personalities I happened to enjoy learning from. I have nothing against automatic forecasting when it is i) absolutely necessary and ii) based on algorithms I can find sound - otherwise I follow D.R. Cox advice in his comment on Leo Breiman's "two cultures" paper in Stat Science a few years ago.
+1. Some related terms that it may be worth the OP's time to look up are "Transfer function models" and "Time series regression models"
Thanks. That makes complete sense. I guess I was looking more for weird probabilistic constructions where the full set of order statistics is not sufficient for the parameters of the distribution of iid data.
Thank you for your answer. Maybe it's correct, but I'd like to read a more detailed (and natural) explanation of this. I have read descriptions in books and slides, but most of them are not very clear and there are slight differences.
Thanks, that's very useful. How would you take into account the number of polled people?
This seems like the best option in reality, only I cannot actually compare a linear model to any actual experimental data. Or did you have something else in mind?
Ok. It would make sense to add some description to the answer. The plot is pretty opaque by itself...
Thanks for pointing out that Likert scales are usually aggregated within subjects. This is the case for me too. I have edited the question above.
@marco try package [segmented](http://cran.r-project.org/web/packages/segmented/index.html). You might search this site also, the structural breaks came up several times in last few months.
I've rewritten your question a bit to try to make it a bit clearer what your questions are.  I hope I haven't changed the sense.  Just re-edit if I have.
If organization is your population and you surveyed all or nearly all its members then you don't need any significance test since any differences then are absolutely significant.
Yeah, I just figured out the `expit` function will get me `p` http://en.wikipedia.org/wiki/Logit. I'll edit the question. But still, which design is better?
$E[A^2] = (E[A])^2$ only when $A$ has _zero_ variance, and is thus effectively a constant. So, No, you cannot assume that the MVUE of $\\sigma^4$ is simply the square of the MVUE of $\\sigma^2$.
I believe it's a test for linear trend within levels of ordinal IVs, generally in a regression or GLM context.
With a large enough sample size anything can be significant...
I'm afraid I don't have any firsthand recommendations for you. On-the-fly data fusion, dynamic data fusion, and, of course, 'data fusion' should help if that's the route you're going to take. Again, I'd like to qualify all this by noting that there isn't anything to fuse on one side, at least not to my eye. Although, if you have information on subsets of your donors (which seems unlikely in your case), that would be closer to your true goal. Best of luck.
I have downloaded the source and worked from there.  However, the values creates by the density function kindly provided by @Cyan do not match those kindly provided by Thomas Wutzler in his logitnorm package. Any suggestions?
What does "strongest" mean in terms of models? Having parameters that don't contribute much explanatory power in your model doesn't usually cause much harm (some inflation of standard errors, when there's multicollinearity may be an issue) - but removing predictors that don't achieve significance certainly can cause problems.
Plot the response against age over the range you're considering if you don't know what a partial derivative is.
The problem is that the ICC uses parametric tests..
Re: "I think of Monte Carlo as any method that uses probabilistic sampling of data as input into a computation..." this is a useful way for me to think of the problem. Thank you.
Would something like summing the squares of the differences between F and G (or the absolute value of their differences) over all the values for which you have outputs for both suffice as a measure? With 6+ dimensions, I'd suggest not using a lattice-type grid of values, by the way, but random draws from the space of values (though you might use quasi-random sequences instead I suppose).
In the usual models where censoring arises (like survival models), there's generally no requirement for censoring to be simultaneous.
Note that you can write the denominator as $\\beta+(y-\\mu)^T\\Sigma^{-1}(y-\\mu)$.  note that the degrees of freedom is equal to double the power of this term minus the dimension of $y$ which gives us $v=2\\alpha+n-d$.  thus we can write posterior as $$p(y)\\propto\\left[1+\\frac{1}{v}(y-\\mu)\\left(\\frac{\\beta}{v}\\Sigma\\right)^{-1}(y-\\mu)\\right]^{-\\frac{v+d}{2}}$$  this is the kernel of a student t distribution.
@FrankHarrell In your handouts, you specify that `orm` is made to handle continuous variable. You also talk about log-log, complementary log-log, cauchit, probit and logit ! Knowing my variables (as described above (including the fact that I have more than 100 levels in my response variable)) which method would you suggest me ?
(+1) Thank you for emphasizing (and analyzing) the dependence of the answer on the model assumptions. If (in the example) you cannot assume anything, then *all* you can say is that the third quartile exceeds $16$. If you do assume a model, then at least you can say to the consumer of your advice, "if your picture of the income distribution is at least roughly what I have assumed, then you can use my result as a reasonable estimate of the third quantile." (Most statistical conclusions are implicitly conditionals of this sort.)
Thank you Nick. I've thought about my data and it is definitely censored, not truncated, so I think censReg is what I will need to use. I made a tobit model with the VGAM package (using this tutorial: http://www.ats.ucla.edu/stat/r/dae/tobit.htm), and the diagnostic plots looked better than ever before, in all my attempts to model this dataset. Which is great! However I'm having trouble figuring out how to do a tobit model that includes a random effect (random effect is site, which data points (fields) are grouped by). Do you have any advice? I'm on the hunt for a tutorial that describes it.
@sara Please, don't forget to register your account, vote on response(s) that were helpful (or not), and accept the one that fit with your original question.
I found the code in this excellent primer on GAM in R - http://www3.nd.edu/~mclark19/learn/GAMS.pdf
The code: library(car)
scatterplotMatrix(mydata[,c(1,1:8)],pch=19,cex=.5,reg.line=F, lwd.smooth=1.25,
spread=F,ellipse=T, col=c('gray60','#2957FF','#FF8000'),
col.axis='gray50')
Although the independence do not hold for me, I assumed that maybe this was something that could be corrected for using Newey-Wests standard errors or something similar.
@Glen_b I don't know how to do that. Could you explain in an answer?
Is $X_k$ one of the predictors you want to include in the model? It sounds like you're saying that $P(Y=1)$ is a U-shaped function of $X_k$. A U-shaped curve can often be well approximated by a quadratic function (especially near the valley) - have you considered including a linear **and** quadratic term in that variable into the model?
Here are some references on [linear trend test](http://stats.stackexchange.com/a/9096/930) with [application to genetic studies](http://stats.stackexchange.com/a/8979/930).
Do you want to understand what semi-definiteness *is*, or do you want to know why correlation matrices *must* be semi-definite, or do you want to know what important results are implied by this property?
@user956 - I would disagree with this.  Neither theory nor experiment are "in charge" of the other - they work together.  Sometimes the theory leads to an experiment, we have an untested hypothesis we want to confirm or deny with some data.
Great! This helps. Thanks a lot to both of you.
By definition, any analytic function $f$ in a neighborhood of $0$ has an absolutely convergent Taylor series there. Your argument therefore suggests that the exponential $e^X$ could just as well be replaced by $f(X)$. Is there nothing special about the exponential?
+1 for the importance of plotting and thinking before diving into modelling.
Generally when splitting data we would look at the mean + sd, rather than just the sd.  Consider an IQ test with mean 100 and sd 10, we might consider someone smart if their IQ is > 110 = 100 + 10, if we consider everyone > 10 as smart then we have a genius population.  This may have been what you meant, but it was unclear.
+1, thank you very much for the "frequentist purists" clarification. I fixed the last cat statement.
http://stats.stackexchange.com/questions/23060/confused-by-matlabs-implementation-of-ridge - comments there might contain the answer
If you have a single categorical variable you are effectively doing a one-way anova and the concept of a linear relationship between your regressors and your response isn't well defined since you don't have any way to measure the distance between your categories or for that matter even ordination.
Do you mean using the new `SIMPLAN` procedure? I'm not very familiar with it - but nothing in the docs suggests you can use it for time series. If you describe your goals in more detail it may be simple to produce a simulation in plain vanilla syntax using the `LAG` and/or `CREATE`.
@chl I will update it to make it clearer. I'm just looking for some guidance on interpreting the clustering comparisons, as don't understand what the output means. @mariana's response was very helpful explaining some of the terms associated with this method.
I think you have hit a nice medium between accuracy and technicality in your original answer: keep it up!  You don't need hats, but if you can manage to show how an estimator is distinguished from other, similar-looking things, that would be most helpful.  But please notice the distinction between *predicting* a value Y and *estimating* a parameter such as *m* or *b*.  Y could be interpreted as a random variable; m and b are not (except in a Bayesian setting).
Thank you! I think that line should be 

`dat<-data.frame(answer=sample(letters[1:4],1000,replace=T),question=rep(c('q1','q2','q3','q4'),each=250))`  


This example is very helpful. Just another stupid question: I have a data frame where each column has the name of a question and the entries are the answers given to that question listed by users (the user id's are the first column of the data frame). 
How do I get this data frame in something like the thing you generated?
Thank you very much!
@Tomek For $n \\lt 10$ we can't hope to find good approximations.  Given that the brute force computation is easy to code and executes fairly quickly, the main reason for looking for *any* approximations would be to improve our understanding of the behavior of the sample range.  If that's your situation, the sum in your question can be reduced to a case of the hypergeometric function.  That's a trivial result but it suggests analytical ways of studying the range and creating approximations.
Particularly for more complex systems, Kalman filtering and related methods should be investigated as well.
This question has already been asked at http://stats.stackexchange.com/questions/45897/what-are-the-main-different-alternative-correlation-estimators (but has not yet gotten any answers).  Our site has extensive material on correlation, especially comparing Pearson and Spearman coefficients: see the search results at http://stats.stackexchange.com/search?tab=votes&q=Pearson%20AND%20Kendall%20AND%20spearman.
Can you please describe the cases that you are interested in? Discrete or continuous outcome, combined with discrete or continuous endogenous variables?
Around 100 hospitals, on average around 500 patients per hospital (but of course it's not balanced)
@ Nick & @Sabbe Thanks - we'd have to collect a lot more data though, so I don't think this would work.  I should have said this in the question - apologies.
@curious - my point is that my "corrected AIC" is the actual AIC, and the thing you are getting from the AIC function is wrong when you are comparing transformations of the "dependent variable".  The point is $-2\\log (p (y|\\theta)) $ changes under the transformation, $ x=g (y) $ (for eg,$ x=log (y) $).  You have to account for the jacobian of this change when using AIC.  The `AIC()` function you are using does not account for this.
Duplicate of many posts. eg [this](http://stats.stackexchange.com/questions/93540/testing-equality-of-coefficients-from-two-different-regressions), or [this duplicate](http://stats.stackexchange.com/questions/99416/test-if-two-normally-distributed-random-variables-are-equal)
Thank you -- this is just what I was looking for!
Henry, thanks a lot for clarifying this. My example in the first data is just an extreme one. I have a lot of other entries which aren't as obvious as this one. I of course do a multiple testing correction after this. And yes, I do a chi square test in cases where the observations are >=5. Thank you so much!
I note generally that the nonlinear least squares fitting presumably used by @JamesPhillips is very sensitive to initial values for the parameters. 3048944 My guess is that you are focusing too much on small details in the residuals. No model for this dataset, or most others, will catch all the detail.
Another option to consider is latent factor modeling. If your four measures of insight are related, you could model these relationships as latent factors and test for group differences in them. If there's only one common factor, this would reduce the need for familywise error correction. However, it might require a lot of data and study of the methods involved.
The sample size is relevant because the "adjustment" that takes place will be more material as the number of parameters becomes large relative to the sample size.
No downvote, but I don't feel this adresses the question.
I think you just reinvented the standard R function `scale`. And now it looks as though you want us to do something with the `ecdf` function but it's not clear what. `ecdf` returns a probability , ie, something between 0 and 1. It may be that its input can have negative values but its output is strictly non-negative. Why not post a reproducible example and say what you expect to see as the result? That will make your desires clear and lower the level of apparent confusion.
@JeremyMiles as far as I know, MATLAB treats 1 as an event. Though I cannot find any conclusive proof of this online
Best in what sense? Note that your main question and subquestion are completely separate, as a distribution bounded by 0 and 1 cannot be normal with standard deviation 1 (regardless of what the mean of the latter is). There is much literature on how sample means can be a poor method for heavy-tailed distributions, but for arbitrary distributions on [0,1] not much can be said. For normal distributions there is no reason not to use the sample mean.
@user11852. Yeah matrix is sparse and PSD
Additionally my predicted curve isn't getting any smoother as i adjust lenght scale. The prediction of the target points always stays the same whereas the prediction of the unknown points gets smoother. But i want the "interpolation" of the given points to be smooth as well when i set a large length scale
@gung thanks for the edit, i have realized how bad my initial wording was, lol
[IRLS](http://en.wikipedia.org/wiki/Iteratively_reweighted_least_squares), which works in the spirit of your current procedure, often does a great job.
Some of your expressions sound alarming in that they reveal your confusion over the topic of factor analysis. `factor analysis on SPSS using varimax rotation `, `factors seem to be uncorrelated according to the anti-image matrix`, also you may be mixing up principal components with factors, as many do. Certainly, you should read more about FA before you start doing it.