tttjjj/elt_bench huggingface dataset:
@Frank Harrell what I gave we're reasons that the standard deviation may be a useful measure of change. Your comments pointed out situations where it may not be relevant... I am not disagreeing with you nor am I advocating for the standard deviation as the ideal metric- all I was offering was plausible reasons why one may use a standard deviation as a metric. To the extent that the  Gini index is discontinuous or highly skewed the standard dev would not be the ideal measure although chebyshevs inequality always provides some intuitive, albeit coarse, interpretation of a standard deviation
Furthermore, there is really no analytical advantage to using the standardized coefficients.
@JonathanThiele it is perfectly possible to have S<P. For example, sets II and IV from Anscombe's Quartets demonstrate this behaviour. Also, see this question: http://stats.stackexchange.com/questions/27127/what-does-it-indicate-when-the-spearman-correlation-is-a-definite-amount-less-th
@user11852: I am not looking for estimate on the same subject. I do repeated measures on various subjects in order to get population estimates. I do not care for the subjects that I already tested as I already have the experimental answer... I want to be able to predict how a new subject of a specific group will respond to the stimulus. Greg answer solves the problem indeed.
I just want to show a 2D-plot with $U_1$ in the x-axis and $U_2$ in the y-axis...then plot the support for $f_{U_1, U_2}(U_1,U_2)$ one one graph and another 2D-plot with $U_1$ in the x-axis and $U_2$ in the y-axis...then plot for the support for $f_{U_1}(U_1)f_{U_2}(U_2)$...tell me what part you do not understand...the reason I asked that ... it is easy to from the support to show that they are dependent
@ttnphns, the job satisfaction variable is a single Likert item w/ only 3 levels. I fail to see how treating that as interval data is the best way to go. Logistic regression seems reasonable.
@act00 See here for a good noninformative prior without MCMC : http://stats.stackexchange.com/questions/33382/how-do-i-calculate-a-confidence-interval-for-the-mean-of-a-log-normal-data-set/33387#33387
Thanks again for these thoughts! I will see in the developement of my model which degree of detail is necessary.
Aww, those comments locks; $\\frac{\\hat{\\sigma}}{\\sqrt{2n}}$. At least this one gives the result in agreement with bootstrap.
Please consider accepting some of the answers to your previous questions.
The reason that the confidence interval will be wide is precisely because you might get an outlier. But t-test still assumes samples are from a normal population.
I hardly know where to begin. Let's start with getting some context for the question and understanding the meanings of the terms you use. Precisely what do you mean by the "probability" of an outlier? Are you supposing a sequential experiment in which the chance of an "outlier" changes over time? If so, what is your model for this? Do you perhaps mean your *estimated* probability of an outlier? If the latter, what was your initial estimated probability (or basis to assess a *change* in probability)? How are you estimating the probability?  How are you identifying "outliers"?
I think $P(U≤u,U=X) = 0$ when $U\\ne X$ and $P(U≤u,U=1-X)=0$ when $U \\ne 1-X$, so I do not see the error. What error are you seeing?
Thanks for the edit gung.
@John, I tried (I've added the output of anova(), see my updated post), but since this is `glm` and not simple `lm`, mean sum of squares and p-values are missing and I don't know how to interpret it...
1) Tukey invented many tests. Which one are you talking about? 2) What are the beta and alpha here? You should be explicit because there are potential ambiguities.  3) Whichever of the numerous possible things you might be talking about with alpha and beta, are you sure the beta and alpha will remain the same if you change the time interval?
I don't understand. As I already mentioned, convergence in KL divergence implies convergence in distribution, know? So whereever information theoretic CLT applies, usual CLT also applies. Moreover, information theoretic CLT also assumes finite variance. Or am I missing something?
Thanks for your most interesting answer, it has opened a whole area of research. Some good references aren't behind annoying paywalls, like the one you provide or [this delightful article](http://hal.archives-ouvertes.fr/docs/00/07/58/32/PDF/RR-0720.pdf). I don't have a solution for my specific case, but I have found that an exponential of the form $880 x^{0.4}$ is a suitable upper bound.
What is the outcome in your specific example? Price? And what do you mean by "w.r.t. index"?
If you can get your hands on a copy of _Categorical Data Analysis_ by Alan Agresti, chapters 1-3 has a very "classical" treatment of contingency tables and measures of association in them.
I would imagine a lot of advice you get would be equally applicable to simulation studies designed to evaluate competing methods of estimation or prediction.
" Is it a typo of "t-test or ANOVA"?" yes - this was a typo - my apologies   - as you suspected it should read "t-test or ANOVA"
Okay. I'm feeling stupid now. That works. I always thought the default was ncol(a) and not min(nrow(a), ncol(a)). My bad.
I think this is wrong. Anything Bayesian is just a fancy version of likelihood. If likelihood fails, Bayesian fails. If correlation between errors leads to biased frequentist estimates, no amount of Bayesian computational trickery will rectify that. On the other hand, Bayesian versions of instrumental variables that can deal with correlated measurement error issues are difficult to set up, unless you want to do non-parametric modeling of the error term distribution.
If that is the case, do you know what your true distribution should be?
@Max: Please let me know if more explanation would be helpful.
@C.Reed: Not with the same ratios ! Sometimes you want to reduce the computation time and/or control the test-to-training-ratio to study the bias-variance tradeoff. I you have some time to spend, I recommend chapter 3 of Kohavis Phd Thesis discussing this topic extensively (http://robotics.stanford.edu/~ronnyk/teza.pdf). But all in all, the cv (from my point of view) is the favored method for error estimation (as mbq already said)
Welcome to the site, @beginnerstat. I wonder if you meant to say, "***two*** standard deviations of the mean"? In addition, I'm not sure I see how this wording improves on what the OP has read elsewhere. Would you like to elaborate a bit?
@jonsca Since we were using boosting methods, which work by combining many "weak" learners, we decided to use any features that could be easily calculated that could provide some benefit. All that is required of a weak learner for it to be useful is that it can classify at greater than chance levels. The AR coefficients are equivalent to a compression of spectral envelope, which gives some notion of the short-term information complexity of the music within that window, although only very loosely.
@January Many thanks for the expanded answer and much appreciated - I believe I now understand the situation. Not necessarily 100%, but it's there. And I'll do some "back of the brain thinking" on it and I'm sure everything will become crystal clear. Unfortunately I don't have enough reputation on this forum to indicate that this answer is useful - _any one willing to substitute an up-vote for me?_
@Raskolnikov: Thank you for pointing that out. I have edited it.
I cant use a Mann Whitney as I only have one group of participants.. However, a binomial test may be better but I was told to use a one sample t-test.. Is it bad to have 1-performed task incorrectly, 2 - performed task with minor mistakes, 3-performed task accurately? I could collapse the levels to just 1 and 2.. I am just unsure of what is statistically correct..
@SpookyFM The second half of my answer [here](http://stats.stackexchange.com/questions/106073/statistical-power-of-roc-auc-test-with-non-iid-samples-to-how-many-iid-sample/106391#106391) addresses some non-obvious issues and proposes a few solutions with using mixed models for risk prediction.
That's valuable work.
If you literally want such a probability (which is eminently reasonable) you need to supply additional information in the form of a *prior distribution* (over a set of possible distributions). Then Bayes' Theorem does the rest. There are other approaches to addressing needs similar to yours. For instance, a "non-parametric prediction interval" uses data drawn (independently) from a *totally* unknown distribution to erect an interval having a specified chance of including $x$. That doesn't involve computing $P(x)$, but perhaps it addresses your needs. Which approach do you want to take?
How do you measure accuracy?Model A has higher adjusted R2 and highly significant (see p-value)
@Chase +1 for including the plot
That's great, and it's faster than my code. Maybe I'm asking too much: is there a way to have the polynomial terms ordered the way I want (i.e. first linear terms, then quadratic and finally mixed terms)?
"I lost points" only indicates that the person allocating points thought it was non stationary. Do you have some convincing *explanation* as to why you should agree? (NB: Are you sure you transcribed everything correctly? Please double check, especially your variance.)
Thanks for that helpful link.
Groups are treatment groups for a training supplement. Group 1 whey protein, group 2 casein protein, and group 3 placebo (control). the pre and post testing are either side of a 6 week training period. I think the ANOVA style that marius has answered might be right, iv just not leaned how to report the observations of the spas output. If you know of any example reportings of this style that would be amazing. on the output, the "test of within subjects effects" box has an additional line to what i have seen before, and I'm not entirely sure of how i would interpret the results.
@IrishStat Actually, I don't know whether this is a "change points" detection problem or critical points estimation, I found this approach just out the link I've provided in the previous comment.
Depending on your goals, OLS may not have to satisfy any assumptions!
Glen, what is the $\\sigma_k$ parameter in your answer?
Because a complex number $x + iy$ is--by definition--just the real ordered pair $(x,y)$, every one of these questions except the last is answered at http://stats.stackexchange.com/questions/19261/multi-dimensional-goodness-of-fit.
Any? Okay, well [here](http://stats.stackexchange.com/questions/74705/official-name-for-symmetric-percent-difference-function-x-y-maxx-y) it's called a 'symmetric percent difference'. But that term seems to mean different things in [other places](http://www.ncbi.nlm.nih.gov/pubmed/11113946). To be honest, I don't think there's going to be a widely used name for it.
I was surprised as well that the newspaper would simply cut and paste (with citation, at least) the facts and figures with no commentary added.  It reads more like an advertisement for the cited blog entry.
Welcome to the site, Susie. Please try to keep your comments/answers constructive - there is no need to criticize the OP for not knowing terminology or concepts, regardless of how basic you think it is.
Excellent answer and a reference I have visited previously. Also worth noting that many report a 1-tailed confidence interval in the case of 0 estimate; just need to be very clear in reporting.
1. Do you have samples from multiple years of each subject? 2. Is Y expected to covary with X?
Hint: $\\mathbf{Y} = \\mathbf{RX}$ where $\\mathbf{X} = (X_1, X_2, \\ldots, X_n)^T$ is a normal vector with multivariate normal distribution $N(\\mathbf{R}\\mu,\\mathbf{R}\\Sigma\\mathbf{R}^T)$ and away we go!
Check out the Pandas library if you go with Python.
I doubt that there can be any equivalence. `mclust` presents normal-model-based EM clustering, wherease k-means is not dependent on any type of the distribution, it is not model-based. These are different algorithms.
You might want to mention that you're working with panel data, if that is still the case. This both complicates the problem, but may allow you to estimate this.
(...) What happens to a more complex case of lasso, where we know that betas are dependent? (copy/paste since the last sentence was lost when converting to comment)
Based on that plot, almost no evidence of seasonality and it is evident you have non-stationary data.
Because I didn't know the answer to this question, I decided to simulate the data, by repeating each experiment with a fair coin (Pr=0.5) to give me an estimate of how frequently 1,2,3,4,5 or 6 out of six times the coin tosses result in a distribution that differs significantly from the expectation. I will update my question above with a specific example, as I agree that I've been unclear. But to answer your question, yes, my expectation is a fair coin (Pr = 0.5) and the number of tosses is sample & experiment-specific.
@Glen_b [Some people don't worry at all](http://xkcd.com/882/). :-)
I'm confused by your question. If you're using a Gamma glm, where do log-odds come in at all? Log odds of what? What is your response?
A bit of both! Thanks Dirk. I had been using : http://www.r-project.org/search.html but this is a much better resource. Thanks
+1 Yes, the real problem with conventional hypothesis testing is that it answers a question that you are not really interested in having answered, i.e. "is there significant evidence of a difference?", rather than "is there evidence of a significant difference?".  Of course what is really desired is generally "what is the probability that my research hypothesis is true?", but this cannot be answered within a frequentist framework.  The misinterpretation generally arises from attempts to treat the frequentist test in Bayesian terms.
Not sure what you are now asking. That book unsurprisingly gives the same basic advice as you are getting. The only small point is that it is incorrect to imply that autocorrelations are always damped. If the true series were, in R terms, `c(1,2,3,1,2,2,3,4,3,2,4,1,3,2,3,1,1,2)` then your series with outliers gives some stronger autocorrelations. The advice that a correlogram 'may be seriously affected' does not mean that you can read off the effect of the outliers from the correlogram.
(ctd)... Or, depending on what you needed to get out of it, you might simply allocate a probability to each point, or simply allocate that proportion of each point. By comparison, a Bayesian decision-theoretic approach would go further by starting with a loss-function (e.g. a function giving the relative cost - in whatever sense of the word you wanted - of misallocation of the point)
`anova(lm(yield~variety+block))$"Pr(>F)"[1]` or any number of other ways of accessing a particular element of a vector that's a particular element of a list. See `str(anova(lm(yield~variety+block)))`
Even with 2 or 3 prices, you would still not have enough variation. For example, if you had fifty cents and a thousand dollars as your two prices, it would be better to dichotomize your price variable (1 = one thousand dollars, 0 = fifty cents, or vice versa) as there is no information for if the price was one dollar or $100.00 (there are very large and distinct gaps between these price options) and there is no ability to distinguish price among these gaps with only two to three price options
As McKinnon, Krull and Lockwood suggest, mediation and confounding are statistically identical. Conceptually is how they differ. 

"Unlike the mediational hypothesis, confounding does not necessarily imply a causal relationship among the variables. In fact, at least one definition of a confounder effect specifically requires that the third variable not be an "intermediate" variable..." - http://dionysus.psych.wisc.edu/Lit/Topics/Statistics/Mediation/mackinnon_PS2000.pdf.
The first half of this is missing, i.e. what your scientific problem is, how Date features in your analysis, why you are treating it as a factor, etc.
@JJM, looking at your additional code, try changing the last line to `dfimp2=dfimp[dfimp$overall_tumor_grade!='X' | is.na(df$overall_tumor_grade),]` - sorry I don't have time to test this or look in detail, as I have to go to bed now for an early rise (here in UK) but I'll check back on this question tomorrow.
Yes, when the loss function is specified externally to the model you can solve for truly optimum cutpoints.  But be sure that you allow the loss function to vary from subject to subject to reflect individual subject differences in opinions.  That means that no two subjects may have the same optimum cutpoint.
@Chase - I didn't ever start on this.  If you'd like to take a shot at it, I'd be glad to help out.
Statistical voyeurism? And there we were wondering what to call the site...
*Please let me know if you have any questions* -- you need to have questions first. You didn't post any.
@Glen_b I can really think of no good example when a specification of intercept-through-origin makes sense otherwise. Even in very practical situations, such as analysis of stopping distance, or chemical compositions after applying a catalyst to some substrate, intercepts that fit through non-zero origins can help to handle measurement error, calibration issues, timing problems, etc. etc. In the years I've done analyses, I've always seen reason to fit intercepts even when the values they take on don't make sense.
Could you please have a look at the [tag-wiki info](http://stats.stackexchange.com/tags/self-study/info) of the `self-study` tag? Your question seems to be a standard textbook type question and would seem to fall under the tag (which covers such standard-bookwork questions whether for a subject or your own private study; while it mentions homework, it's not limited to homework). If - as it seems - the tag applies, please add the tag. (The same goes for your other questions.)
I don't see exponential distributions here.  The plots indicate you have mixtures of distributions, each of which may be symmetric or close to it.  So the first thing to do is a robust ANOVA (such as [median polish](http://www.itl.nist.gov/div898/software/dataplot/refman1/ch3/median_p.pdf)) and investigate its *residuals*.  This may show that ANOVA is fine.  Otherwise, it will suggest ways to re-express the data that leave symmetrically distributed residuals.
I'll add to @julieth: I wouldn't get excited about any "accuracy" measure unless a high-resolution calibration plot accompanies it, and the plot (say, based on loess) is penalized for overfitting using the bootstrap.  The R `rms` package makes this easy to do for some problems.
*Side note: be aware that correlations ( / the lack thereof) can be largely driven by the extent ( / the lack thereof) of the range of the features. This fact may mean that stronger ( / weaker) correlations don't imply what people will intuitively assume.*
@whuber I am waiting at chat.
Welcome to the site, @Florian. I'm not entirely sure I know what your question is, can you clarify it? Are you asking if the measures on the left are the correct descriptive statistics to use? (If so, what is your goal in providing descriptive stats?) Are you asking if the formulas on the right are the correct formulas to get the descriptive stats listed on the left? (If so, this question would be off-topic for CV--see our [help page](http://stats.stackexchange.com/help/on-topic).) FWIW, eg, `STDEV.P` gives the *population* SD, ie, it is a *biased* estimate of SD.
Already answered [here](http://stats.stackexchange.com/questions/34636/interpretation-of-simple-predictions-to-odds-ratios-in-logistic-regression/34638#34638).
@Cardinal How would you recast this as a linear program?  It's not evident how to do it even in trivial cases (such as two data points and one parameter): there are no constraints and the objective function is nonlinear.
$(n+1)/2$ is the median of the $n=n_1+n_2$ ranks. Note, too, that the ranking procedure preserves the signs of the residuals: when a value exceeds the median, the rank of that value exceeds the median rank, etc.
The statement "you don't really have multinomial logistic, you have ordinal logistic" doesn't make sense. The ordinal logistic model is a submodel of the multinomial logistic. What you've said is like saying "You don't have a polygon, you have a square".
It is a time series if the data are collected over time.
Yep, sorry I wrote quickly. I meant: Consider the model $y_{ij}=\\mu + \\alpha_i + \\varepsilon_{ij}$ or $y_{ij}=\\mu_i + \\varepsilon_{ij}$ and highlight graphically the following decomposition: $y_{ij}=\\bar{y_i}+\\varepsilon_{ij}=\\bar{y}+(\\bar{y}_i-\\bar{y})+(y_{ij}-\\bar{y}_i)$, that is obs. are expressed as deviations from the grand mean + group mean + fluctuations around group mean. Then, we have $(y_{ij}-\\bar{y})=(\\bar{y}_i-\\bar{y})+(y_{ij}-\\bar{y}_i)$ or total variation = between-group variation + within-group variation (which is basically your first picture).
Thank you, Niousha. By including "also" you are indicating your answer somehow goes beyond @Martyn's [suggestion last August](http://stats.stackexchange.com/a/68590/919), but you seem to be saying exactly the same thing.  How does your answer differ?
Strictly, G is the likelihood ratio test *statistic*.
Are set A and B two correlation matrices of the same elements?
Even if i made the gender question optional, if for some reason males are more likely to respond than females doesnt the error remain?
Regarding your flag on the appropriateness of such question here, it was closed as off topic because there's no statistical issue. It could also be considered too localized. However, feel free to reach our chat room, [Ten Fold](http://chat.stackexchange.com/rooms/18/ten-fold), and ask around.
@ars - note that the "linearity" of the 'true' model is not the only way to recover the 'true' model from a linear model.  If the non-linear component of the 'true' model can be well modelled by the noise term (e.g. non-linear effects tend to cancel each other out) then we could reasonably call the linear model 'true' I think.  This is similar to assuming the remainder in a linear taylor series is negligible.
*Is* Logic a branch of Mathematics? Including three-valued logics & modal logics, or just first-order predicate calculus? Are all formal sciences somehow Mathematics?
That would seem to be a reasonable explanation.
I see your point. So with my approach I basically assumed that the variance was constant and specified. If I want to estimate it, I need a prior on it. $\\mathbf{P}(\\mathbf{\\Sigma}) \\sim \\mathcal{F} (\\mathbf{\\mu_{\\Sigma}} , \\Sigma_{\\Sigma}) $
Now, my problem is that it's not clear how to define it, and what would be an appropriate distribution for it, but this seems to be out of the scope of the first question.
+1, nice write-up
I forgot to mention that the measures of bias & discriminability in this approach are on the log-odds scale. Thus, the estimate of the intercept is the log-odds of responding 'old', the estimate of the main effect of truth is the increased rate of responding 'old' (in log-odds) induced by old-ness, etc.
I'd be extremely wary here to apply any kind of standard error calculation in this context, because there are 2 sources of variance here (model instability + finite set of test cases), and I think resampling validation will not get around the finite test set variance: consider cross validation. In each run, all test cases are tested exactly once. Thus variance between the runs of iterated CV must be due to instability. You won't observe (nor reduce!) the variance due to the finite test set this way, but of course the result is still subject to it.
There exists hardly any difference between the results of the two (see Paap&Franses 2000)
more than "not significant" -- the variance is actually estimated as zero.  Looking at `summary()`, or `VarCorr()`, would show this.