tttjjj/elt_bench huggingface dataset:
Again, I suggest benchmarking the MC results against the manual calculations using the full permutation (re-randomization) test. See the [code for your example](http://pastebin.com/ZjiQ8A2B) for a comparison of `oneway_anova()` (always close to correct result) and `aovp()` (typically far off from the correct result). I don't know why `aovp()` gives wildly varying results, but at least for this case here they're implausible. @gung the last call to `oneway_test(DV ~ IV | id, ...)` in my original answer specified the error strata for the dependent case (different syntax than used by `aov()`).
@babelproofreader: http://cs229.stanford.edu/materials/ML-advice.pdf  Here is a paper with 2 examples on pgs. 7-8.
Thank you for the post. I totally agree with you
Thanks! I encountered the same problem when skimming Dempster's paper.
In fact, it does follow from (296) in the Matrix cookbook by taking $x = (f(t), f(s))$ and $A$ the block matrix with three zero blocks and $\\Sigma$ in the top right corner. The variance-covariance matrix of $x$ is given in terms of $\\Sigma$ and $\\Omega$ using the notation from @mpiktas's answer below.
For an explicit example of this situation, see the analysis of 10 IVs at http://stats.stackexchange.com/a/14528.  Here, *all* the IVs are strongly correlated (around 60%). But if you excluded all of them, you wouldn't have anything left! Often it's the case that you cannot drop *any* of these variables. This makes the T&F recommendation untenable.
The question is not about how to identify or exclude outliers: it is about how to estimate the parameters of the "background" distribution. Although the two may be related, a lot more work needs to be done once you believe you have an outlier detection method: then you have to assess the bias and efficiency of the resulting parameter estimates (presumably based on the dataset without the outliers). Please notice that most of the contamination distribution will *not* be identified as outlying because about half its values will be less than 3SD above the background mean.
How large is your sample? Are all the observations matched (Jan-Dec)?
It is simply telling you that it is not possible to have that high a correlation between the two series.  Because they have such different AR coefficients, even if you fed in identical error sequences they would wander away from each other all over the place.
The intuition is clear: the more data you have, the less you have to rely on your priors. Not just a statistics lesson, but a life lesson! ;)
+1 A subtle point is that even the appearance of the "$d\\theta$" in the integral is *not* part of the likelihood function; it comes from nowhere. Among the many ways to see this, consider that a reparameterization changes nothing essential about the likelihood--it is merely a renaming of the parameter--but will change the integral.  E.g., if we parameterized the Bernoulli distributions with the log odds $\\psi=\\log(\\theta/(1-\\theta))$ then the integral would not even converge.
No model can ever be reported as "the truth". As George Box said "all models are wrong, but some are useful". The models answer different questions; which question do you want to ask?

Also, adding additional variables will always increase $R^2$, although not always by that much.
It is unclear from your description, whether `A similar B` pertains to the state of reality (i.e. how close they are to be the same entity) or to the degree of measured match.
@Xi'an Sorry to bother you again, but your definition of `h` contain the t distribution, when it should just have the $\\sqrt{|x/(1-x)|}$. This causes all the estimates to be off because you are multiplying by f^2 instead of f.  The approximations should be around 1.015, while they are around 0.25.
By "percentage" you mean "accuracy"?
Important nuance: those are not blanks, but periods (stops), which in Stata means missings, meaning here "can't be determined".
One perfectly viable option is to select several bandwidths and plot the results, picking one which looks good.  Basically, if you reject methods such as LSCV because they appear to oversmooth or undersmooth, you are sort-of doing that anyway by choosing the intermediate mechanism of a bandwidth selection methodology that gives a good-looking result (instead of just choosing the bandwidth itself.)
@RobertF Time obviously is critical, but just as obviously, allowing SB to consult a calendar trivializes the problem. Here is what one philosopher has written about the role of time in this paradox: “[Y]ou have gone from a situation in which you count your temporal location as irrelevant to the truth of H, to one in which you count your own temporal location as relevant to the truth of H.” (Elga, Adam (2000). Self-locating belief and the Sleeping Beauty Problem. Analysis 60 pp 143-7.)
Correlation doesn't tell you how similar two sets of ratings are - if one set of ratings are all 1 or 2 and another set of ratings are all 4 or 5, the two sets of ratings are very different (the ratings are 'not similar', so presumably the similarity between users is low), but if the movies that get a '2' rating from the first user are the same ones that get a '5' from the second user, their correlation will be 1.
For reference: I don't think that this needed to be community wiki.
Hi @Theja, I updated the question. Yes I use the same classifier (e.g. a gene signature) for all methods and dataset. The classifier is taken as input by methods as dataset.
http://stackoverflow.com/questions/16312201/add-missing-columns-to-an-r-table  Do i need to keep this here or better to erase this?
Although you describe a *procedure,* you haven't yet told us what it is for.  I am concerned because there are several statistically suspect aspects of this procedure; likely you need an alternative.  Could you back up and plainly explain to us what you are trying to learn about these data?
@Besct, if AdamO's answer resolved your confusion, you can thank him by *accepting* it. You do this by clicking the check mark to the left of the answer below the vote total.
@RussLenth To be clear niandra's (now) presented the *two-sample* Kolmogoro-Smirnov test, which is a test of whether two samples are drawn from the same distribution, as, I think, you are asking for.
You might want to take a look at the `heckit()` and `selection()` functions from the sampleSelection package. See [link](http://en.wikibooks.org/wiki/R_Programming/Tobit_And_Selection_Models)
In the notation I borrowed:  x_t := log(X_t) to the p_t are the logs of the P_t.
@GungIt seems that the data is categorical. May be  ANOVA could be used if translate the number of records  into the proportions.
@Erik, I don't see a problem with high number of cases?
Anthony, you've answered my questions on two different websites now. Thank you!
You need to state what software package you are using.
Because actual parameter values are a matter of convention, and conventions vary, please define your dispersion parameter or at least provide a link to the definition you are using.
Thanks, my test data has 192 entries and 26 characteristics.  Also, the difference I am trying to expose is between two distance functions using a single linkage method.  If I give my user the choice between two distance functions and they give the "same" result, this may be alarming to my user and it may be an indication that I should limit the choices in the case of Ward linkage.  On the other hand, this could be an indication that I made a mistake in my implementation.  By the way, when I say "same", I mean the clustering is the same, but the depth of the clusters is different.
The Cost of Forecast Error has been discussed in the practitioner-oriented journal _Foresight_: http://forecasters.org/foresight/ Very much recommended! (Full disclosure: I'm an associate editor.) I agree that the CoFE is not readily apparent in your example, but then I would wonder how much effort you should really spend in optimizing your error measure...
Are the random variables mapping a (fair) coin toss `(Head -> 0, Tails -> 1)` and `(Head -> 1, Tails -> 0)` identically distributed? Are they the same functions?
If you can evaluate the distributional shape, is there a reason you can't compute the median, even approximately?
the middle term should be $e^{Bz}$
Are you assuming the fit is done with least squares?  That would be a poor approach indeed, given that the residuals don't even have zero expectation!
In R you can see `lme4` and `nlme`. Both have been discussed here a lot and both have a lot of documentation as well.
Jerome, Although I cannot read @penguin_knight's mind, I suspect he was referring to your intended audience and not to you.  After all, you did not ask for an answer; you asked *how to "explain this".*  He may be suggesting that you might already have succeeded with a mathematical explanation in your question statement and that it's sufficiently simple to be persuasive to most people.
You might find the following threads helpful to start with: [What is the meaning of p-values and t-values in statistical tests?](http://stats.stackexchange.com/questions/31/) (note that F is analogous to t-values here), & [How to interpret and report eta squared / partial eta squared in statistically significant & non-significant analyses](http://stats.stackexchange.com/questions/15958/).
Afaik gini impurity is used for classification Problems. Nevertheless, the question refers to the size of the trees.
@Nevedita: I added an example for `auto.arima()` - `arimax()` should work very similarly. I hope this helps.
Also  my dataset has 3 categories which are the store id and the data spans several months. Would you say a better approach would be to start with less months and/or separate model for each store?
@micans its the R^2 ;)
There are books written on this topic. The one I use is Harrell's "Regression Modeling Strategies." He criticizes N-fold cross-validation as an inefficient use of data, and suggests (as well as offering the software to implement) that you use bootstrapped methods.
How do you define the "time of operation" if it's not the time of the processes where the operation is used? Is it the time of the process divided by the number of operations used in that process or...?
I don't think you really need to do any normalization. Clearly your means and variances are already in the same magnitudes.
sorry, I just realized that these two terms give the same results. So I guess they are the same :-)
If you scale a predictor $x^* = cx$ in a multiple regression then both the mean and standard error of the coefficient will be multiplied by $1/c$.
I can only agree with chl's answer. Perhaps it can be helpful for you to have a look at the Psychometric Models Task View at CRAN: http://cran.r-project.org/web/views/Psychometrics.html
I haven't see much evidence that using STAN results in much better inference than JAGS in many situations. And for these simply conjugate models where JAGS can figure out the blocking structure and do moderately intelligent updates there doesn't seem much point to me. Last I checked STAN required all sorts of weird tricks to get working on mildly complicated models, and even then it doesn't necessarily work (e.g. the rejection rate for a Bayesian neural net was absurd when I tried it).
I recommend Wilcox's book as a general stats text that uses R (see my answer http://stats.stackexchange.com/questions/25632/what-book-is-recommendable-to-start-learning-statistics-using-r-at-the-same-time/25876#25876). Although he doesn't cover off Fisher's exact text, there's enough detail on the specific text on the web which will make more sense if you've got the background in that book on similar tests.  There might be a better text out there on this specific sort of goodness of fit problem but I think Wilcox's book is great as a general intro that quickly takes you to a high level.
See the edit to my comment; your original has the advantage of being able to use any of the 9 definitions of quantiles. Then again, `boxplot(quantile(x))` would work in place of fivenum and probably matches the original post better; I'm just used to associating boxplots with fivenum.
`why I have not seen an example yet where they actually calculate the variance covariance matrices in each class`. I wonder if you have been simply missing it. The covariance matrices need to be computed, and Box's M test compares them.
Could you provide additional criteria that would help narrow down the possible distributions?  What, for instance, is the reason for "generating" this distribution?
While this link may answer the question, it is better to include the essential parts of the answer here and provide the link for reference.  Link-only answers can become invalid if the linked page changes. Please also include a full reference for each.
thnx for you answer , but what should i do in this case ??
A method advised by Tukey as a way of *flagging* "outliers" was to compare their values to 1.5*IQR above and below Q3 and Q1 respectively. With 25% of your sample consisting of "outliers", I don't think this method (or other methods) will be reliable. I generally don't advocate automating such processes since data can be weird and "outliers" can dramatically increase your power to detect trends. If you have data quality issues, *flagging* outliers can be good practice, but you may want to focus on feeding in higher quality data and reviewing it more closely.
No, wait, actually that wasn't my question. I didn't compare the method="pearson" with the method="spearman" version of x and y. I compared `cor.test(x, y, method = "spearman")` with `cor.test(rank(x), rank(y), method = "pearson")`. These estimates will be identical no matter which data one chooses.
Thank you nonetheless! :)
Building on this, even a graph like what they use for high/low daily stock prices could work well.
Is $\\pm 0.67448975$ any more useful?
@gappy, did you consider using QR (or Cholesky) decomposition for matrix $\\Omega^{-1}+BD^{-1}B^T$ (the middle term in Woodburry formula)? The remaining operations are simple matrix multiplications. The main source of instability is then the calculation of $\\Omega^{-1}$. Since $m<<n$ I suspect this application of QR or Cholesky combined with Woodbury will be faster than QR on all matrix $A$. This of course is no state of art, just general observations.
You forgot to add a ' to one of the $x_i$'s.
Welch invented what is now called Baum-Welch algorithm (he call it "the easy part"); Baum proves mathematically that algorithm works ("the hard part"). See http://courses.cs.tamu.edu/rgutier/cpsc689_s07/welch2003baumWelch.pdf for exact details.
That's not true ... The second paragraph of `?pvalues` says "In the list below ... those marked (+) support GLMMs as well as LMMs"
@PeterFlom True. But under normality (which is the standard assumption about the random effects in multilevel models), independence and zero covariance are synonymous. The point is: If the variance component ends up being estimated to be zero (or essentially zero), then this is the same as just leaving that variance component out of the model. I would not suggest to do this a priori, but if that's what the data are showing, then you will get the exact same results whether you use the multilevel model or just a standard linear model.
@RobertKubrick Is the lower plot with the V shaped fitted Ys the result for the robust regrssion?
Perhaps the absence of a standard test item indicates that you are correct, and it is more appropriate to use whatever plot is simplest in the system being demonstrated.  That makes it hard to compare, however.
+1 This site needs more "basic" questions like this.
I presume you mean with one case it is either positive or negative.
@MichaelChernick: nice example. So you argue that hospital site should be treated as a random and not as a fixed effect. But what would be the actual difference in the outcome between these two options? If we treat it as fixed, then we will get a regression coefficient for each hospital, and can test e.g. if the main effect of hospital is significant. If we treat is a random, we will not get a regression coeff for each hospital (correct?); can we still test the main effect of hospital? More importantly, does it maybe increase/decrease power of other main effects/interactions in the model?
Are both variables ordinal?
For monthly wage data in logs (I presume it's the natural logarithm) take the exponential of each monthly wage. Then sum the monthly wages by year. That way you get your annual wages. Taking first differences of those gives you the growth rate of the annual wages.
Very well said - I was hoping you'd find this question!
*Something* is assumed to be normal in regression, but it's not the DV itself (and it's *certainly* not the IV's).
For every $\\theta_j$, $1/\\theta_j$ will also work, won't it? But people usually restrict themselves to invertible models, which would restore uniqueness.
To take differences you need more than ordinal scale.
Thanks a lot, I know there are 2 variants of single exponential smoothing. Nevertheless I do not understand what the goal of this model is: to predict the next value $Y_t$, is it? But this model should be used for prediction of time series without trend when the observed data is a sum of level and stochastic components. The latter is impossible to predict so what is left?
Thank you again for useful comments on software. I work with Mathematica
5 studies, all of them roughly enrolling 100 patients. Odds ratio resulted just above 1; I used fixed effects model. Yes, I made forrest and funnel plots.
Is your normal equation right? I would have thought: `normal = @(x, mu, Sig) (1/(Sig*sqrt(2*pi))) * exp((-1/2)*((x-mu)/Sig)^2);`? Not that it affects your charts
I described the problem in such an abstract way because I thought it was something applicable to any situation regardless of the background. To be honest, I still believe so; although, no doubts the more information one has, the better results s/he can produce. I was looking for some best practices, just like the ones that you have shared. Thank you.
I really don't know. The problem would be to evaluate the predictive covariances. I've never seen this problem before. When I read the OP question, I tried too see what kind of answer a nonparametric model would give us.
As I said in my post, predictive power is of primary concern. Having sensible explanatory variables is desireable but not a priority. Therefore, if $Y = b_0 + b_1x_1^{.21} + b_2x_2^{.73}$ gives me better lift then is acceptable at this stage. The question is how to choose the best set of transformations to give the maximum lift.
I don't get this example. dat$y only has 2 levels -- which is too few for a mixed model.
Please see http://stats.stackexchange.com/questions/71127/one-sample-case-test-on-means/ The concept is the same. I suppose this could even be marked as a duplicate.
actually it goes through it in less detail, but with code examples
+1.  It is interesting--and quite telling--that the readings start and end with Meehl.
Hi,the book is this
5 levels                                               Scores<-c(49,34,45,59,74,53,51,57,77,65,71,85,79,91,60,72,50,41,65,57,62,47,59,48,37,26,52,39,44,50,41,35,88,70,95,77,81,89,62,83)

Degree<-c(rep('Business',8),rep('Science',8),rep('Liberalarts',8),rep('Finearts',8),rep('Engineering',8))

Aptest<-data.frame(Scores,Degree)
mat = matrix(c(0,0,1,-1,0,0,-1,0,0,1),ncol =2)              mat
     [,1] [,2]
[1,]    0    0
[2,]    0   -1
[3,]    1    0
[4,]   -1    0
[5,]    0    1

my.contrasts = mat %*% solve(t(mat) %*% mat)
Is this really a statistical question? It seems more a "what Excel function do I use to do X?" question (even the function itself doesn't seem to be very statistical). Please identify how this falls under the [subjects that would make it on topic](http://stats.stackexchange.com/help/on-topic) for CV or edit your question to make it more clearly on topic. As it stands I think this question is firmly off topic here and should be closed (many questions nearer to being on topic are closed quite quickly). ...(ctd)
Exactly. That F-test tests whether the fixed effects are zero. Rejecting this hypothesis means that they are not zero.
I know latex (a bit) but how do i use it in the forum?
What do your acceptance rates look like?
Nice answer -- welcome to CrossValidated, Joe. Your other answer on Quora are excellent and I am glad to see your contribute here.
I didn't understand about the different sets of units - units of measurement or experimental units (two groups of individuals with A, B, & C measured on each individual)?
@zzk: exactly my point above. While it works for the normal distribution, it does not work for the log-normal distribution! This is of course an interesting problem, how to find a joint distribution on $\\mathbb{R}^p_+$ with a given covariance matrix, however I do not know of a closed form solution.
Are gender and anxiety are correlated? What happens if you do the analysis separately by gender? What size N do you have? Are plots of the data helpful?
If all you know are two means, mean (average) cost and revenue per click, then no statistical test is possible. You can just think about what the difference means in business or economic terms.
I don't know enough about ANNs, but unless the activation functions differ substantially in shape it will be very difficult to tell them apart. For a discussion of an analogous situation, you could see my answer here: [Difference between logit and probit models](http://stats.stackexchange.com/a/30909/7290).
Ok, so, you see how sample sizes and calculation formulas explain the difference in df you reported in your initial question. -An effect on what? Strongly asymmetric distribution of subsample sizes can produce an artifact referred to as 'Simpson's paradox' but the asymmetry you describe does not appear that extreme.
OK, it works. Here is the Java code: `double totalDemand = 0; for (User u : tour.getUsers()) {totalDemand += u.getMean();} double cumulativeProbability = new PoissonDistribution(totalDemand).cumulativeProbability(50);`
@Tal Yep, your "sign test" is also known as the Wilcoxon test.  The Mann-Whitney U test is its unpaired counterpart.  The *Mann-Kendall* test is its extension to more than two sets of data.  Although it doesn't seem to be commonly known, the U statistic can be obtained as the t-statistic for the ranks of the *combined* dataset (using midranks for tied groups) and the Mann-Kendall test is really just ANOVA on the ranks of the combined data.  This insight allows you to avoid the complex formulas often cited and use simple software to do the calculations.
I'll give it a try. Though I cant accept the answer since it doesnt answer the question..
The "easycalculation" formula is grossly wrong: it confuses the value of the *probability density* with the *complementary cumulative density*.  (See the [Wikipedia article](http://en.wikipedia.org/wiki/F-distribution) for instance.) There is a moral here: do not rely on unattributed formulas you find on the Web!
-1 This is clearly wrong. You may want to read the higher voted answers first.
How well-versed are you with time-series analysis? I'm not an expert on ts, but looking for regular periods (often called [seasonality](http://en.wikipedia.org/wiki/Seasonality)) is fairly standard.
Is this homework? If so, it should have the homework tag.
@CP I like your Bayesian approach. You also justify you name!
Rousseeuw and Ruts have another article online besides the American Statistician one [in postscript format](ftp://ftp.win.ua.ac.be/pub/preprints/97/artbag.ps) for free.
@chl: I assumed that getting the filenames into a variable was his main question.
I noticed a flag for migration to SO, but I think there really is a statistical question beyond R code.
I understand your first question means softare that "finds the unspecified determinstic structure" . AUTOBOX a product that I am involved with as a developer (http://www.autobox.com) is one. A second is SAS and a third is SPSS. I don't understand your second question. AUTOBOX allows the user to specify alpha levels (probability levels) to guide selection of trend vs step. For example one could set .05 for steps and .10 for trends , tilting