tttjjj/elt_bench huggingface dataset:
Roughly, each term $(O-E)^2/E$ behaves (asymptotically in the maximum likelihood theory) like the square of a Standard Normal distribution. There are $k$ of them, so *if they were independent* their sum would approximately have a $\\chi^2_{(k)}$ distribution (because that's what a Chi-squared distribution is!) and if they were subject to $p$ linearly independent linear restrictions, they would have a $\\chi^2_{(k-p)}$ distribution (because such sums act like sum of $k-p$ *independent* squared Standard Normals). So, asymptotically, it is obvious what the degrees of freedom need to be.
@whuber I don't care much for the distribution, as long as all allowed values have a finite probability. I think that using your $d_i$ doesn't ensure that $n_i=1$ whenever $m_i = 0$.
Thanks for your answers guys. This is perfect. I think it really is to do as to whether there is a "bumpy" or seasonal pattern involved or whether you can consider time as reltively smooth. I guess if you're doing a panel of many economies or units with highly differentiated features (for example, developed and developing countries) the series of time dummies might make sense.
Any way to move it over automatically?
Thanks Andy! That's really helpful :]
Aren't the larger degrees of freedom present in the "protected" t-tests going to by their nature be anti-conservative?  I guess it will be a subjective issue whether departures from sphericity will be sufficient to reject the pooling of variances. I guess it is also open to decide whether to use those pooled variances while doing other post-hoc procedures.  Honestly, I don't care to defend a difference I found or one I did not.  I'd like to accurately present the data, but short report journal articles don't always give enough space to tell the whole story.
Thank you! The original article is here:
http://www2007.org/papers/paper676.pdf,
page 5 (839), right before the 5.2 heading. I hope the technicalities of the field (network science) won't stand in the way of discussing the statistics issue.
@gung I'm not so sure about that: couldn't this question alternatively be interpreted as asking whether the model expressed in the `aov` command is appropriate for the given data and stated objectives?  As such the question looks to be on topic here.  Yes, it could be improved by restating it in a less `R`-centric way, but heretofore we have allowed many questions (and answers) that were expressed entirely in `R`-like language, for better or worse.
The only thing I would do differently from that is use a different bandwidth. The default bandwidth for density() is not particularly good. density(x,bw="SJ") is better. Even better would be to use a bandwidth designed for mode estimation. See http://www.sciencedirect.com/science/article/pii/0167715295000240 for some discussion.
If you take two samples at precisely the same point, will you get the same value of f() or different values due to the error term?
@Filip, I'm migrating this for you.
@sds I updated my answer for score explanation.
Hmm, good point. I tend to think of Fisher with low cell counts only, but using it in general for 2x2 tables makes sense now that we have fancy computers =).
How different are they? Is it a matter of the number of decimals printed/displayed in the result? Or is it fundamentally different values? Are the versions of R and all packages the same (checkable with `sessionInfo()`)? Could you give an example of the different results or, additionally, the code which generates the results?
Are you interested in finding a general difference in agreement or are you interested in testing each individual question?  How many levels in each scale?  Do the questions test different things?  Are they mutually exclusive (i.e. rankings that must all be different like a sort)?  Please provide more details on what you intended in your study.
R and S-Plus are very similar, but definitely not the same (I know from painful experience!).
You ask for a conditional probability. *Ergo*, the only elements that need consideration are those appearing in all the sets.  Accordingly, you don't have to worry about $n$ sets: you just have *one* set; namely, their intersection. At this point objective analysis of your question ends, because it makes no sense to provide a probability (except in trivial cases where all or none of the elements of the union have attribute $t$) until you describe *how* a given element in the union was selected. What can you tell us about that procedure?
(+1) Good suggestions. I'd like to harp on though, that the model's "inaccuracy" is merely a failure to predict many probabilities over 50%. If the "1"s typically have predicted probabilities of 10% to 40%, compared with a little under 0.5% for the "0"s - that would be considered strong predictive performance in many applications.
For a next time, it would be better to ask two questions.
That would be tritomizing if the word existed [dichotomous = di (2) + tomous (cut)].  Collapsing categories is, on the average, a way to make the model less effective.  The only time I have collapsed $Y$ is when there are two high-frequency categories that are out of order with repect to more than one $X$.  I pool those categories.  There are papers formally studying the pooling of categories.
Ok I will adjust, thanks
@Wayne Those are just two different examples, ive posted real world examples now
Immediately comes to mind. Similarity between `abc` and `adefd` would be `mean(aa+ad+ae+af+ag + ba+bd+be+bf+bg + ca+cd+ce+cf+cg)` with corresponding proximities from the table substituting these paiwise combinations.
1. Please give the full reference for the book by Erna Weber. 2. At what level do you want to calculate the confidence interval? 95%? Because now, you only add and subtract 1 standard error from the estimate whereas for a 95%-CI, it should be something like `exp(r-1.96*se)`. 3. How were the estimates in the book obtained? By a generalized linear model, like you did?
Thanks! Works like a charm!
Transformation to what purpose/to achieve what? As it stands your question is unanswerable.
Sorry about commenting in the wrong place. Live and learn.
Oooppss sorry, my previous claim is true only under the hypothesis of the "null model" (intercept only). Otherwise the distribution of $R^2$ should be something like a noncentral Beta distribution, with a noncentrality parameter involving the unknown parameters.
@Ray, thank you for being attentive to spot a lapse.
+1 to Steven. @Tobias, modelling the noise in the observations is going to be important here as you have several different values for y for x = (8,8).  covNoise basically helps you to model the conditional variance in y.
How does right skew lose 'the log-linear relationship'?
Great answer.  I don't get how you're supposed to answer these type of questions effectively without providing almost the full solution.  If you don't want an almost complete solution then be VERY specific with the question.
Hi thank you that is a great reference. :-) Do you know of other sources that would go into detail about how to go about scaling and rescaling?
(+1), this is interesting, but note that eigenvalues are no more ordered than test statistics are. The real problem is that the eigenvalues are typically not independent before they are sorted.
It only takes an additional moment after the correlation.
Can somebody help on this, i am stuck
(+1) For the first sentence. However, Taleb is more of a (self-absorbed) polemicist than a serious intellectual. I only have the first edition of BS; his commentary on statistics is overstated and uninformed in many places, but the attempted thesis of the text is more than what's quoted in the first sentence, as you point out.
I would say, a clssical case of [click me](http://stats.stackexchange.com/a/739/442). As everyone said: You should probably get more data.
@Michael, you seem to be showing the converse. A minor detail, but important in light of the question, is that even when arguing in the direction you have, the conclusion $X=Y$ is "only" true *almost surely*.
the documentation (http://www.mathworks.com/help/matlab/ref/chol.html) indicates that `p=0` means `A` is positive definite. For your second question, note that a Gram matrix ($AA^{\\top}$) is positive semidefinite, and you can always 'boost' the eigenvalues of any matrix by adding $cI$ to it; for sufficiently large $c$, $cI+A$ will be positive definite.
Hi, but this equation above only suit one dimension.
+1.  Just thought it was worth mentioning that the Poisson is a special case of the Negative Binomial and that one way of deriving the negative binomial is as a mixture of many different Poisson distributions with different means.
Do you have a set of "rules" by which you define a return as correct or not ?
The Statistical sleuth is used a the textbook on that great introductory course (there are 64 lectures in total)  http://video.google.com/videoplay?docid=-3474013489970580510&hl=en&emb=1#
Against your intuition. This is clearly not uniform distributed. :D
Why do you call this "retrospective" ?
@Macro Ok that's clear. My question is more about the standard errors.
One note besides. The link says `...the relationship between y and x is non-linear, while correlation and covariance are measures of linear dependence between two variables`. This is not quite so. Pearson correlation is, but not covariance. Because the latter is sensitive to the variance magnitude it is not a measure of exactly _linear_ dependence.
for the decreases, I think it is due to something I forgot to mention in my post. A, B and C all contains NA values, so when I added them to the model, the number of observations decrease from 149884 to 62213. This might have caused the decrease of the conditional r-squared. Anyway, mod.0 is here because I want to show the r-squared computed with the r2.mer methods. If we take a look at mod.1 only, do you think it makes sense to report the effect of A,B and C, even though the marginal r-squared are so small? With such large sample, significance is actually not of surprise.
@ChamberlainFoncha : Which two models do you want to compare? It is probably best if you edit your question to clarify which models you want to compare and why you want to do that.
Thanks For Your Reply.But I have one query that by using this formula may be we will not be able to get best number of cluster.Is there any way to find best number of cluster.Please suggest.
Since I presume you'd only want the constraint to be applied when the process is white noise, the answer is evidently no; you'd need to know the process is white noise in order to activate the constraint, in which case there's no point in doing the estimation (and therefore no point in having the constraint.)  Also note that you won't in general end up with $\\hat{a}+\\hat{b}=0$ from an estimation routine, due to noise in the series.
Regression is a way to approximate given data. Hence, approximation is more general concept because there are many different ways to approximate data or functions. Other approximation methods are, e. g., polynomial approximation using, e. g., the Lagrange or Newton polynomial.
To have a larger sample size between thresholds.  Instead of each point in the ROC being estimated by a single comparison, it is estimated by 100.
It's great to see someone thinking about the statistical analysis *before* carrying out the experiment.
Welcome to the site. Nice answer. It would be even better if you could provide a cite.
A standard quick approach is to divide the data into thirds along the s axis (with the intention of comparing the first and last thirds with a test of central tendency, usually a nonparametric one).  If you do this for the 5..30 data and produce three side-by-side boxplots, the trend will be clear and convincing.  The middle one, though, will *not* suggest a logarithmic tendency.
Dorp, J. R. van, & Mazzuchi, T. A. (2000). Solving for the parameters of a beta distribution under two quantile constraints. Journal of Statistical Computation and Simulation, 67, 189–201.
...which is the Robert & Casella (2009) book mentioned by Stéphane Laurent in a comment above.
Roughly speaking, bias is "mean error", but as @abaumann mentioned, "mean error" is not the same as "mean squared error".
I don't have access to the article, but this argument indicates a rather flawed understanding of what's going on.  Despite a flawed understanding, the conclusion that other statistics are worth consideration is reasonable.  Expected loss function is simply an estimate of the expected *value* of the loss function (e.g. squared error, logistic, etc.).
You mentioned each survey was already weighted. Are they weighted using similar methods? Or, did the particulars of the surveys necessitate using different weighting methods?
@PSchnell I am chiefly interested in the former, though information on the latter would also be appreciated.
For logistic regression [it's fine](http://stats.stackexchange.com/questions/67903/does-down-sampling-change-logistic-regression-coefficients/68726), though you'll have to be careful that the sample of NS firms is not biased with respect to any predictors.
(+1) However, Ripley is wrong on the point where he says that the models must be nested. There is no such constraint on Akaike's original derivation, or, to be clearer, on the derivation using the AIC as an estimator of the Kullback-Leibler divergence. In fact, in a paper that I'm working on, I show somewhat "empirically" that the AIC can even be used for model selection of covariance structures (different number of parameters, clearly non-nested models). From the thousands of simulations of time-series that I ran with different covariance structures, in none of them the AIC gets it wrong...
@JCL Perhaps, they can if they see the table. But, what if there is no table? Then a table can be re-constructed to some extent using the text which refers to such things as 'grand means', 'marginal means' etc. I am guessing the phrase 'marginal mean' is a literary convention to signal what we are talking about. In a similar vein you could argue that why use the term 'sample average' when 'average' is sufficient. Qualifying words such as 'sample', 'marginal' add some context.
Feel free to provide links to Wikipedia, but at least provide a sentence or two of explanation in your answer.
Well you need to have some means of rating the result of the combination of the 5 performance measures, otherwise how will you know if you combined score used to rank the performance measures is effective? Failing this, all you can do is combine the 5 measures using a summary statistics such as the mean or reduce the dimensionality using e.g. PCA.
@Eleonora, please peruse these threads: http://stats.stackexchange.com/search?q=%2Binteraction+%2Blogistic+%2Bregression and, if you still cannot find your answer there, perhaps you can edit your question about what _specifically_ you can't find and it can be reopened, if appropriate.
Genetic epidemiology routinely uses $\\alpha=5\\times10^{-8}$ in genomewide association studies, often regardless of the precise number of tests performed.
I don't see why this is being called "nonlinear regression." It is a time series model. The power k for X$_t$$_-$$_1$ could be like a Box-Cox transformation in a linear model with the power being a parameter that can be estimated by maximum likelihood.  With k as a parameter it is nonlinear in the parameter k.
This question doesn't seem so bad to me. A value that tiny for the determinant strikes me as suspect to to a *very near* ill-conditioned  correlation matrix as the input for `FACTOR`. So you might reorient the question to something along the lines `The determinant of my correlation matrix is *really* tiny - should I be worried about the results of factor analysis?`. That seems to me to be on topic here. Posting your correlation matrix would help others with there assessment.
thanks Nick,I am new in this forum and my English is bad. I will improve it if I have another question in the future..
@wolfies This is all the information I am given.
Why Fisher's test?
(cont) in which case you are probably right in assuming the model isn't very good, since it doesn't care about the input predictors at all. Then again, maybe this is correct, maybe these are highly uninformative predictors, but without any data you simply don't know.
No, they're (plainly) dependent in general.
Can you paste the entire regression output in your question?
re Edit 3: Given $(x_0, y_0, z_0)$, $r$ is easy to find.  To obtain $(x_0, y_0, z_0)$, Newton's Method should converge rapidly from some reasonable starting value obtained as in (3).
It is pretty hard to not fit the Bernoulli distribution unless you have correlated observations.  What about the fit do you suspect is inadequate?
@dominic Frequently--especially when reading papers--a histogram is just about the most detailed presentation of the data available.  This situation is not new: older statistical texts spend a lot of time on making estimates from histograms and binned data.  For instance, see the [MathWorld article on Sheppard's Corrections](http://mathworld.wolfram.com/SheppardsCorrection.html) and note the dates of the references.
Good idea. But Im a bit dummy for Python.
How do I split variables?
I doubt that capture recapture is suitable for human beings ;-). Nevertheless, the question is good. Here in Switzerland, every few years there is a large survey on health issues, including delicate questions. But in other countries, I dont't know if there are similar data available.
(1) The Spearman (rank) correlation can be computed just fine even with non-discrete data.  (2) My point about machine numbers was that viewing data as discrete or not is a modeling decision, whence it is unwise to base one's statistical judgment solely on whether, *for some other purpose,* the data have been considered discrete.  (3) Your second paragraph is difficult to follow because you seem to be advising against the use of sample statistics to make any estimates at all.  (It's unlikely the OP has a complete census anyway.)
@chl:  very handy, I hadn't seen it.  Thanks for pointing it out!
@user603 Unfortunately, no.  I'm sure I've seen the problem before, though.  Maybe on a problem set in grad school.
Depends on your effect size, standard error, etc. Read about sample size calculation, power analysis, and study design.
Thanks a lot for your comment. Yes, I'm following the Duchateau and Janssen's book and trying to write function for EM algorithm, really its a critical work. Hope SAS macro will be helpful.
Good. The first three are nearly all the maths I know. And the fourth should be read together with Casella & Berger - very different emphases.
Although the idea is a good one (`xlim` and `ylim` are parameters that `acf` passes on to `plot` and they will indeed limit the plotting window), this code is not even syntactically correct.  Following the two-line example in the question, `acf(dummy, 20, xlim=c(1,20), ylim=c(-0.2,0.5))` will work.
Hi...I do not know what you mean with "conducting a permutation test with it to compute p-values"...any suggestion?
Is this problem for a class?
As far as I have understood the OP, his issue is NOT the measuring of performance of a classifier (which is hopefully a good estimate for the future expected performance) but how good one can get AT ALL, i.e. what is the maximum (not absolutely per metric (AUC is 1 at max or something like that), but for a given problem)
Nice Python code by the way!
This is very close to this existing question on when to use Spearman versus Kendall http://stats.stackexchange.com/questions/3943/kendall-tau-or-spearmans-rho
This question does not have a unique answer: the answer depends on how the data were obtained.  From that one can develop a probability model, allowing for (say) the prediction of the true values represented by a, b, and c, *conditional* on knowing their sum equals 500.  Perhaps, then, to make this question answerable, you could fill us in on how these values were obtained and what the purpose of adjusting them is?
I am puzzled: because a hypothesis is a statement about reality, it refers to fixed, non-random parameters.  *Ergo*, $a_i$, $b_i$, and $x$ are *numbers*.  Consequently $X = b_i/a_i$ is also a number, not a random variable: there is nothing to test!
@user603 It's actually the most basic property of monotonic transformations, period: it follows directly from their definition and assumes nothing about $L$ itself.  After all, if $L(\\theta)\\gt L(\\theta')$ for all $\\theta'$ (that is, $\\theta$ maximizes $L$) and $x \\gt y$ implies $f(x) \\gt f(y)$ whenever $x$ and $y$ are both in the domain of $f$ (that is, $f$ is monotonic), then by letting $x=L(\\theta)$ and $y=L(\\theta')$ it is immediate that $f(L(\\theta))\\gt f(L(\\theta')))$ for all $\\theta'$ (that is, $\\theta$ maximizes $f\\circ L)$.
+1 Very nice answer. The part of your contribution--which is informative, well explained, and well illustrated--seems to boil down to the fact that any reasonably smooth function (such as the plot of eigenvalues *versus* their rank for a large correlation matrix) will, by the very definition of "smooth," look close to linear when one focuses on any short section of it. You have established that the linearity does not hold throughout, thereby putting an important qualification on the O.P.'s observation, and have characterized the general behavior very well.
+1. Just to add/clarify: AIC (and AICc) employs KL-divergence. Therefore, exactly because AIC reflects "additional" information the smaller it is the better. In other words as our sample size $N \\rightarrow \\infty$, the model with the minimum AIC score will possess the smallest Kullback-Leibler divergence and will therefore be the model closest to the "true" model.
Thank you user603 for your help.
Thanks for this.  I have been asking around, and my personal network of statisticians doesn't know beyond standard survival analysis, so I appreciate the help.
`is different in that it computes ranks on all the data, not just the pair being compared` Maybe here is the root. There was recently a [question](http://stats.stackexchange.com/q/50917/3277) about t-test/ANOVA, apparently similar. It was about whether to estimate variance from the two groups being currently compared or only once from all the groups.
I believe [Mills Ratio](http://mathworld.wolfram.com/MillsRatio.html) may be relevant.
So, if I don't have the exact form of the probability distribution, I can binary-search the histogram table for such leftmost and rightmost bins where the number of population is just barely below the given threshold?
It would be helpful if you could share sample/dummy data.
sorry I have taken along time to get back to you on this. I am already using a random effect model with subject as the random effect. What I am concerned about is that since in the prisoners dillema ones payoff depends on the others payoff is there not some non-independence issue there? apologies if this is what you are explaining.
+1 This is a good answer.  What makes chemical data different from these general recommendations, though, is that frequently these data are highly intercorrelated: one chemical test may yield a vector of results (concentrations of various analytical parameters), each of which is a slightly different measurement of some related characteristic of the system being measured.  Thus, some multiple-comparisons adjustment is needed, but Bonferroni may be far too strict.
And just to clarify - doing a post hoc test (which one?) on degree will compare each degree to every of the seven other degrees and show whether each pair is significantly different from each other?  (That's what I'm hoping to do).
i don't quite understand what you mean by the "logit part". you might want to look at this review: http://www.jstatsoft.org/v27/i08/
Probably not without a lot more details about your survey. Are the answers intended to indicate the same construct, or different ones? How much data do you have? Are the "etc." factors continuous/counts/ordinal/nominal? Did you have any kind of experimental control, or would you be content to predict even without being able to make causal inferences?