tttjjj/elt_bench huggingface dataset:
No problem.  Thank you for correcting my spelling!
+1 Nice answer!
@Stat I'm very serious when I say that I can't conceive of a situation in which it would be impossible to simulate data from a model. At the very least, bootstrapping from the training dataset qualifies as a valid simulation approach.
Maybe I miss something in your reasoning, but can any transformation formula be of any value, given that you plan to _rank_? Also, what relation tag "rank correlation" might have to your question?
do the answers to this question apply? http://stats.stackexchange.com/questions/7004/calculating-required-sample-size-precision-of-variance-estimate/7008#7008
I've updated the title of your question to reflect its content. Otherwise, it is a duplicated of [Graphical data overview (summary) function in R](http://stats.stackexchange.com/q/4089/930).
Thanks. You note that Frequentist tests are "typically easier to apply". Do you know of a specific situation when this is not the case (if F are only "typically" easier, there must be some such cases). For the present, we can set aside issues of which approach would be "safer", & just concentrate on the *easier to apply* part.
A multilevel model wouldn't mean that you have 74 equations, you would have 1 equation w/ a random effect. You *could* get 74 BLUPs, but you don't have to.
Don't have a reference but if the population size is N=N$_1$+N$_2$ where N$_1$ and N$_2$ are the respective population sizes for the two groups and the true proportions are P$_1$ and P$_2$ respectively then the difference of the 2 estimates in the case of independent samples has variance P$_1$(1-P$_1$)/n$_1$+P$_2$(1-P$_2$)/n$_2$ where n$_1$ and n$_2$ are the respective sample sizes from the two subpopulations without the finite population correction. with the finite population correction you would have [P$_1$(1-P$_1$)/n$_1$][1-n$_1$/N$_1$]+[P$_2$(1-P$_2$)/n$_2$]][1-n$_2$/N$_2$].
That doesn't rhyme in Australian :D
onestop provided an answer that will hopefully give you enough to go on. If you *really* want to know about the distribution of the sample correlation coefficient itself, then the definite reference is: Hotelling, H. (1953). New light on the correlation coefficient and its transforms. Journal of the Royal Statistical Society, Series B, 15, 193-232. Note that this isn't light reading.
There are some good starting points if you google derivation of poisson distribution, that show how Poisson is magically derived from binomial distribution where n is large and the chance of an event is small.  From there it starts to make sense to use it to model count events.  The question I guess is how well do real count events match that smooth extension of the binomial situation.
Actually, the data is ordinal, so an ordinal test is in place, and Spearman is. I believe that with a lot of ties, or as a test of correlation, Kendall's $\\tau$ correlation might be better, see [here](http://stats.stackexchange.com/questions/20124/spearman-or-kendall-correlation/21261#21261). $\\chi^2$ throws away the order of the variables, so can be tested to see if this order is relevant.
I'd imagine this refers to testing blocks of variables at a time, where all potential collinearity occurs within a block, which does not affect omnibus tests like the $F$-test or the likelihood ratio test, but I've never heard the term "chunk test".
I think this could be improved with formatting, editing, and the like.
Or which one to start first?
I'd want to make just two comments about your interesting question. 1) Describe in _words_ your approach besides showing its code (remember that people here use various software, not necessarily R). 2) The scatterplot isn't very much convincing without spikes to its floor. Also, if you have any _specific_ doubts about your approach, please speak them out to make the question more focused.
Could you please also give a complete reference? Links have a habit of dying.
Keep in mind that one byproduct of overfitting can be the gross underestimation of standard errors of parameter estimates.  So you should concentrate on real variances and not apparent variances.
Both of you are perfectly right. I must apologize thoguh. Actually, we did not manipulate a posteriori the coefficients, what we did was to multiply by -1 the observed values of our proxy (being that proxy a rate, we might even change the rate observations into (100-rate), which is a measure that makes economic sense). Is this wrong?
This question appears to be off-topic because it is about how to use R, & not about any related statistical issues. This question would probably best be posed on the r-help listserv.
+1, & you have an impressive knowledge of our politics for a New Zealander ;-).
@Belmont, what is $u(\\alpha)$? Could you provide more context about your problem? Link to article with standard properties of LAR for example would help a lot.
@mbq: I know that how it works, but I need mathematical formula for this prediction error. I have applied Least Angle regression (R package 'lars'). So I am looking for mathematical formula for this cross validation procedure for R function (cv.lars) in 'lars' package.
1) The map itself does not leave an impression that the results are degenerated (oversimplified), but you should try 3d solution, for sure. 2) What is the stress value and what type of stress yo use; how the distances X disparities plot look like? 3) Euclidean distance usually isn't quite informative for binary data; are you willing to pick up a better (for your specific investigation) proximity measure. 4) If that was metric MDS, try nonmetric.
are your permutations properly randomised, if that makes sense in your context?
@gung I'm not guessing the conversions. It is the measure itself that is a guess. For example, I often can't tell whether a *cup* refers to a US cup or an imperial cup (284ml). If I determine that a US cup is meant, then it might not be clear whether a traditional cup (236.58ml) or a 'legal' cup (240ml) is meant.
...before (and after) the actual data of interest is filtered.
@user13655: Exactly! Yes, indeed, the mixture structure can be broken by a Gibbs sampler. Once the data is divided into groups, each component can be analysed in the most convenient manner, eg by EP-ABC if anything else is too hard.
@Roman, good catch :)
@probabilityislogic, ah yes, you are right.
Thanks Patrick! I am interested in the species differences, and I tested several populations of each species in my experiment. However, I am not interested in the 'population' effect, I just want to look at the 'species' effect. By specifying my model like I did, I think I've nested 'replicate' (or 'population') within 'species'. Didn't I?
Great answer.  A plus 1 from me.  I would have liked to give it more.
Crossvalidation seems dicey, perhaps arbitrary and not necessarily very helpful, when doing PCA.  It's not like regression where we might say, "in the training sample these 3 predictors are all effective based on ___ ; are they similarly effective in the test sample?" I don't see an analogous, clear, very testable hypothesis for crossvalidating PCA results.
See http://stats.stackexchange.com/questions/11009/including-the-interaction-but-not-the-main-effects-in-a-model.  The data are incapable to telling you what is true, and such an approach is heavily dependent on the measurement origin for the variables being multiplied.  Assessing isolated interaction effects of temperature in Fahrenheit will give a different picture than if using Celsius.
Welcome -- Nice to see you here, Pat!
Correct - the standard error is minimized at the mean and symmetrically on both sides. I suggest sorting on the absolute value of the t-statistic to address that symmetry. You need a ruler to compare which residuals are more "outlier-y". The magnitude of the residual is not a good measure because it depends on its proximity  to the mean. So I suggest normalizing the absolute residual with respect to the prediction interval so you can compare across residuals. If the residual is large but farther from the mean then this approach will proportionally give that residual more tolerance for error.
Albeit ANOVA comes with stricter assumptions than a _t_ test. ANOVA assumes (1) near normality of the outcome variable (scores in this case), and (2) equal variances of the outcome in both groups.
It would be really great if someone could comment if what I wrote makes any sense ;)
No software dependency issue. It's defined like that in `getLlikAIC`, which gets called by `summary.merMod`. I'd assume there are (theoretical) issues with computing these for a REML fit. (Although `AIC(model1)` works just fine.)
Could you tell us what it means for a valuation to be "standard ... across respondents"?
@Flask, sorry! I answered your title more than your actual question. I edited in some links which might be of interest
[This](http://stats.stackexchange.com/questions/2410/can-cart-models-be-made-robust) question is probably relevant, as is the fact that you can specify your own splitting criteria in `rpart`. See the discussion of the `method` argument in `?rpart`.
Did you mean to say $F(0)=0$ instead of $F(0) = 1$? The reason for asking is that if $F(0) = 1$, then for $t > 0$, $F(t) = 1$ and so the integrand in the definition of $G(x)$ is $0$ and so $G(x) = 0$ for all $x > 0$.
@Michael, regarding your last comment, if $x$ is not symmetrically distributed around zero, there could be large collinearity between $x$ and $x^2$. Since this will probably affect the standard $t$-test for a regression coefficient, a likelihood ratio test may be more appropriate, right?
Great! So the general formulas of paired and unpaired t-test are equal ($$ t= \\frac{\\bar{X_D} - \\mu_d}{\\sigma_d /\\sqrt{n}}$$) - only that for a paired test the sd is $$Var(Z) = Var(X) + Var(Y) -2Cov(X,Y)$$ while for an unpaired one it is $$Var(Z) = Var(X) + Var(Y)$$ ? so using an unpaired test where I should use a paired one (assuming pairs correlate positively) would give me a smaller $t$, thus a smaller $p$ and thus inflated significance and increase my risk of a false positive.
And yes for each experiment, I am simulating from an assumed model of an even coin (P=0.5), but my number of trials = sample size of the original experiment. Maybe resampling is a misnomer?
Do you have a definition for expectation?
I really do not know the rate. To be honest all I know is that my program crashed with a division-by-zero and that I need to handle that case somehow.
Here the 7 values are the nearest neighbors from the whole training set
Thanks, this seems very clever! I'll definitely keep looking into the literature - since any method I choose is going to require as much art as science, having an expert on my side will surely help.
@idoda, coef_ only works for linear kernel, but not rbf. Please see my update in the answers. Hope it helps, thanks
Burnham, Kenneth P., and David R. Anderson. "Multimodel inference understanding AIC and BIC in model selection." Sociological methods & research 33.2 (2004): 261-304.
I have a couple of practical suggestions then for achieving stable and reliable solutions. (1) Use $\\exp(a)$ in model 1 and $a$ in model 2 to obtain comparable results without NaN problems. (2) Run model 2 first and feed its estimates as starting values to model 1. (3) Break model 2 into two parts: only $x_0$ is nonlinear; $a$ and $b$ are efficiently determined by least squares (*e.g.*, use `lm`). You can then use a *one* variable application of `nls` to estimate $x_0$. If you want the full output of `nlsLM`, feed it those estimates as starting values.
I doubt you can bring the expected length down below $\\frac{1}{3}$ for a 50% confidence interval on a sample of 2 items.
Correct, the coeficient estimate from the example is -5.6... not 5.6... I edited the answer, thanks.
As I said, it is probably just the distribution of patterns in the training set being more "difficult" than in the validation set.  I use a lot of resampling in my performance analysis in my research and I see this sort of thing all the time.  The only thing it usually means is that the validation set is too small to give a reliable indicator of performance.  I wouldn't worry too much about it, and just use more resampling to get a more reliable performance measure.
I'm confused as to why you say the KL divergence is not non-negative. Plenty of sources say the opposite (e.g. http://mathworld.wolfram.com/RelativeEntropy.html). You seem to be talking about an approximation calculated using a set of samples, which, yes, can be negative. But I don't think that's what this question is about.
As I mentioned, the subject of research is not preference of users but 'goodness' of products. This method is more about recognizing preferences of specific users...
I have no SO account. I occasionally have problems with the 2l methods for large data, but have never seen R itself crash on in. My guess would be that they are related to sparse data (very small clusters). If all else fails, use pmm with cluster allocation as a fixed factor, and check whether the intra-class correlations of the observed and impute data are similar. The mice package does not use any own fortran or C code.
How can quantile regression far into the tails be robust? It is designed to return whatever quantile you ask for. That is why I emphasised earlier that robust and quantile regression are not to be seen as alternatives.
@FrankHarrell: Well, yes: communication. I spend quite a lot of my working time doing translations: physicist <-> chemist (<-> chemometrician) <-> statistician <-> medical doctor, and I find the art of doing this well has a lot to do with allowing simplifications and finding compromises (and trying to avoid to gross misconceptions). I tackle the "don't throw away information" problem by  sneaking in continuous measures (as chemist I prefer MSE/RMSE) which are well (or at least better) behaved, but their names and purpose are familiar.
Are the samples paired?
Hi Glen, thanks for that suggestion - I suspect that may allow me to answer why $B$ is multiplied by $n$. However, still the rest is unclear to me.
+1 Very well expressed.
Thank for the motivation. This question led me to writing this manuscript:
http://arxiv.org/abs/1304.4920
+1. Though I'm a bit puzzled by "stratified analysis": does that mean analyzing each level separately, with no pooling? (E.g. Doing a regression for women and doing another regression for men?) I've seen "stratified sampling" a lot, but this is the first time I've seen "stratified analysis".
welcome to the site. If you are getting stuck with the log-likelihood, it means your model is not identified. Although nothing can be simpler than your model (which is in fact a random effect ANOVA model, so you could have started from `anova` or `xtreg, re` just as well).
Stackoverflow won't let me post as an answer for some reason, but I cannot reproduce this with current master. What version of statsmodels are you using?
You explain perfect:) thank you:))
Without more detailed information, people will be unable to give you more than generic guidance.  Do you at least have a variable that indicates what kinds of plants are being grown on the farms? That would seem to be a key piece of information which, if missing, might make it difficult to achieve good results no matter how the analysis is performed.
One thing that they do in fisheries is to estimate the population size of a species using capture - recapture sampling methodology.  This is certainly different from what we do in medicine and as you point out sampling methodology is the key to this.
Have you considered the [example in Wikipedia](http://en.wikipedia.org/wiki/Bayes_factor#Example)
Can you get multiple measurements at each frequency?  Or if not, how fine grained can your variations on frequency be (so you have multiple measurements close to each peak, if not on it).
Would you mind providing more detailed explanation, especially with regard to the updated question (A/B testing)?
Do your boxplots take into account the fact that your repeated measures are within individuals?  If so, they are probably pretty useless for letting you guess about the outcome of the mixed model.
I don't understand this question. 

A hierarchical model often implies that a posterior predictive distribution on a parameter is a mixture distribution, so maybe there's some confusion here. 

For a particular parameter, the prior within the hierarchy can be any kind of distribution. However, it's more common to use something simple, like a conjugate prior and allow the complexity/mixture modeling to come from the hierarchical structure itself.
That depends on what you mean by "compare them". What, precisely, is your question?
Could you please explain a little bit more? What is the help function I need to check? Thanks in advance:)
That is the problem. Google returns way too many results and not all the papers or tutorials are clear enough.
I don't think the main question is distinguishing between background and foreground - I think that's pretty straightforward.  The real challenge is creating the graphic in a way that makes it as easy and accurate as possible to make the comparisons that you're interested in.  In my experience grid lines make it easier to read off and compare values - although I don't have any formal experimentation to back this up with.
This kind of question is fairly common & has been addressed a couple of times here on CV. For the best treatments of the underlying issues, you should read the following questions & the answers by Jeremy Anglim: [what-are-good-basic-statistics-to-use-for-ordinal-data](http://stats.stackexchange.com/questions/97/8995#8995) & [is-amazons-average-rating-misleading](http://stats.stackexchange.com/questions/31598/). If you still have any questions left over after having read these, edit your Q w/ what you still need to know.
interesting... you can have two people on her with the exact same handle...
@Dan It would also help if you could tell us why you are doing this. As an aside: doing something like m + z*s may very well end with a upper bound of greater than 255 (depending on your values for m, z and s). Some sense of what you want to achieve (i.e., your goals) would help.
Thank you prof. Hyndman.
I think you meant to say that the 'discrimination' parameter is analogous to the uniqueness parameter in FA, not the difficulty. Also, if IRT (more specifically, the 2PL model) is the categorical analogue for a single factor FA since logit(P) = a + b*theta, then wouldn't MIRT be the categorical analogue to exploratory and confirmatory FA since the logit transformation is still the same; logit(P) = a + b1*theta1 + b2*theta2?
It would help to quote the textbook directly or (if it is available online) to provide a link to it.  Much can get lost if even a single word is taken out of order or out of context.  (For instance, residuals are usually defined as the difference between prediction and response, not the other way around.)
Reading again my question, I see it is not clear in what I actually ask. I know what decision tree and boosting are; I am investigating in how those techniques have been applied in the physics experiment. In particular, what features and classes and statistical measures have been considered to build up trees.
What exactly is the question?  Your title implies that you're getting predicted probabilities smaller than 0 or larger than 1 but your "question" says that the probabilities make sense.  What exactly is the issue?  My inclination is that you might not understand the link function and what it does with regard to a generalized linear model (in this case either probit or logistic regression) and you're trying to make predictions by hand but those don't end up making sense because you're doing them wrong.
Classification error is far too inefficient to use for judging models.  It is optimized by bogus models.  Outside of the normal distribution the whole idea of sums of squares seldom applies.  Think about truncated/censored data, gamma, Poisson distributions, binary logistic models, etc.  We don't use sums of squares for those.  We need a generalization, and the log likelihood is an excellent optimality criterion in general.
it's essentially a duplicate of [this question](http://stats.stackexchange.com/a/49914/603) where the OP wants to compute the Mahalanobis distance of a dataset with singular covariance.
Thanks for this refined illustration.
You should be using a database of papers in your area (Google scholar?), not academia.SE for this.
@shabbychef I updated my answer to include a sketch of an exact inclusion-exclusion approach.  It is worse than hairy.
I think that the interaction (in which case I would assume as the term gonad:nnd) has no interpretative meaning, but rather only tells you if the rate of change in the probability of spawning as explained by the continuous variable nnd is significantly affected by the variable gonad. That is, is the rate of change significantly different across the three categories of gonad or not.:)
I have added code that performs the normal approximation from the paper at the end of my answer (it was too big to fit here).  I get $p(3000, 300, 20)\\approx 0.6468276$ (an error of about 0.0005).
Is this time series financial data, such as stock prices?
I don't follow some of the equations you've written; some things appear to be missing. The key here is to understand the definitions of *scale family* and *scale equivariant* and then to connect the two. Very little is needed in the way of explicit calculations.
I've just exploring my `multtest` package results and I see that I have a subset of the same FDR corrected values `[1] 0.08199 0.08199 0.08199 0.08199 0.08199 0.08199 0.08199 0.08199 0.08199 [10] 0.08199
` So maybe it is as I wrote.
If this is for homework or self-study, you should add the `self-study` tag. See http://stats.stackexchange.com/tags/self-study/info
I am still hoping someone can answer my question, based on further details in response to comments from "onestop", above. Thanks!
A quick comment, without getting into the more subtle issues a more complete answer would: it is clear you cannot calculate the relevant probability from the information given. To have any chance of solving this problem, one needs to know/estimate the SAT scores of those who were rejected (or equivalently, know/estimate the SAT scores of the population of students who applied).
I would like to know What kind of value of MSE/RMSE can be viewed as good prediction?
Welcome to the site, @Lowerison. I edited your code to make it fit in the window & easier to read. Note that there was a typo (extra paren & quote) for the title of the histogram. I tried not to change anything substantive in the code. However, I don't get a good plot when I pass `obs <- c(runif(100), runif(100, min=.5, max=1.5))`; could you check to make sure it's still right?
According to my understanding of autoregressive models and kriging, I believe there are profound differences among them that go beyond some vague difference in "spatial structure." But the meaning of a "spatial autoregressive model" might vary according to what you are reading and what field they are in.  Therefore, it would help if you could explain *your* understanding of what these two approaches are and provide some relevant references to support your answers.  (Your link does not seem to address any of these issues.)
And by the way, logistic regression is equivalent to describing each curve with two parameters - the intercept & slope of the fitted logit.
@Michael Mayer Around 800
What are $w$ and $z$ here? Is $t(\\cdot)$ a known function? What transformation are you talking about that generates index $d$? Can you give some context to the problem? Is this a predictive model for air pollution? Inference on fMRI data?
In general, your weights are directly inversely proportional to the variances. Suppose you have two random variables. If the first has $\\sigma_1^2 = 1$ and the second has $\\sigma_2^2=4,$ then the weight on the first will be 80% and the second will get 20%.