tttjjj/elt_bench huggingface dataset:
It's an artifact due to approximations made by the underlying `R` code; the discrete second derivative operator greatly magnifies the small errors.  This has been discussed in another thread which I cannot at the moment find ...  When you think about it, smoothness of the derivative (of a kernel density estimate as computed in `R`) isn't even defined in this context, because everything in sight is a discrete set of points, anyway.
Welcome to the site, @Curie. This isn't quite an answer by CV standards. We are trying to build a permanent repository of statistical information in the form of high-quality questions & answers. As such, 1 thing we worry about is linkrot. It's fine to have a link as a reference, but not as a standalone answer. Would you mind expanding this to provide an explanation?
I know of no reason why not. EFA is based on the correlation matrix, so, while larger samples may yield more precisely estimated correlations, you don't run into the overfitting problems that you would with regression.
Ah ok, then yes, this solution won't be useful! Btw, if you are doing million times, I guess you are controlling for multiple testing and correcting p-values??
The `cmprsk`-package in R is very useful for competing risks analysis, you might want to look into that.
What do you mean by "random fixed factor"? Also, why do you say "treated as fixed effect or random effect"? That's a big difference; the distinction is the main point of mixed models, are you indifferent to which is used? Can you say more about your situation, your experimental setup, & your data?
Welch originally published in 1938, not 1946: Welch, B. L. (1938) "The significance of the difference between two means when the population variances are unequal", Biometrika 29, 350–62.
Unless I've missed something, I believe $dF(x)=-d(1-F(x))$. However, that's what you need for the integration by parts approach.
There is a power and precision loss if you use a nominal model and you could have used an ordinal one.
It looks like you have two dependent variables.  Do you have both (liquid/solid) observations for the same individuals, or are the people being fed liquids and solids from separate sets?
Honestly, what does that mean?
To have unstandardized discriminant coefficients equal the standardized ones one must scale the predictor variables to variance 1 _within_ groups (did you do that?). But the standardized coefficients can exceed 1 in magnitude. Do not confuse them with regression coefficients beta.
@datodatuashvili, I don't think you can make a general statement like that. The resolution would depend on how many lags you have, length of the series etc.
I guess you could further improve on it by transforming the vectors to norm 1. That way the euclidean distance is [0,2] and the correlation is [-1,1].
I'd just follow most textbooks by saying $t_{critical}$, and declaring my $\\alpha$ as $5\\%$ elsewhere.
+1 for going above and beyond, @Michael Chernick. I certainly got stuck at this exact point: I could walk through a Bayes Rule application, but actually analyzing a set of data is a bit more complex. Trying to figure out how people were doing it with JAGS/BUGS was a puzzle. The book "Doing Bayesian Data Analysis: A Tutorial with R and BUGS" by John K. Kruschke has really helped me and I can highly recommend it.
I'm tring to estimate the cointegration of two historical series(prices) of stocks, I have those two series with date->price with two years history. I'm tring to understand if they are cointegrated and what coefficients they have to calculate the exposure. What steps can i do? thank you so much!
Thanks for this clarification. I really appreciate it. Perhaps the ordering approach only works for classification tree. For your reference, this approach is explained on page 329 of the book element of statistical learning (http://www-stat.stanford.edu/~tibs/ElemStatLearn/)
in fact the measure I had in mind was a simple D'agostino Ksquare but that would only work if I assume that my theoretical dist. is normal...though I want to assume uniform.
Thanks for your fast reply. But no, my question is not about the implementation of the function. I put it here so that everybody could see it. 
As stated in the main post, my two questions are about 1) choosing a p-value after a regression using permutations, and 2) robustness of this procedure, and wheter or not (and how) Bonferroni corrections should be used. 
Regards,
I'm already browsing this book. +500 on this suggestion, EXACTLY what I was looking for: A fat tome full of information about statistics and the appropriate function to use it on R(which is free and open source yay!). Its very inexpensive for the size of this book and its uniqueness. Don't feel scared for being for behavioral sciences. Im using it despite I am a CS student. I found the behavioral statistics books to be even of more help for me than the statistics one to **understand** what is going on. Im not saying a pure statistics one is useless, you need one. I used Paul Meyer (1965).
Thanks for the references: I had read in Littell et al.'s "SAS for Mixed Models" [book](http://books.google.co.nz/books?id=z9qv32OyEu4C&printsec=frontcover#v=onepage&q&f=false) that using AIC might be a reasonable way to choose between two covariance structures **for otherwise identically specified models**
see a rather comprehensive list on [MetaOptimize Q+A -- Good Machine Learning Blogs](http://metaoptimize.com/qa/questions/3163/good-machine-learning-blogs)
It returns a single number, the "$\\beta_{y\\cdot x}$" in the exposition, as explained by the adjacent comment `# Returns the coefficient`. Another clue is that the return value is a quotient of two sums and a sum of an array would be a number.
You can place upper and lower bounds on this probability. For instance, could the information be consistent with a probability of $1$? If so, then it is *certain* that at least one car will be sold each day. A quick check shows that the standard deviation could not approach $7$ (it cannot exceed $12\\sqrt{1/50 \\times 49/50}=1.68$). *Ergo,* the largish SD is imposing an upper bound on the probability. There is a lower bound on the probability, too.
From this document, it appears that continuous variables would be predicted using a normal model. Thus once again I am asking, how is logistic model applicable here? Does IVEware support Pareto distribution at all?
Yes, but it's worth noting that tests always tap more constructs beyond just what they are intended to measure. Eg, an English test (TOEFL) can help predict physics GRE scores for non-native English speakers.
+1, nice answer! Very clear and thorough.
Thank you very much, Dilip. Hanna.
Can we have the link of the answers you mention?
@whuber, in a screening design (full factorial or fractional factorial) with no replication (each combination of conditions is only tested once) the traditional p-values cannot be calculated since you would have 0 degrees of freedom for estimating variance.  Under the null hypothesis that all effects (main and interactions) are 0 the estimated effect sizes will be linear combinations of normal errors (assumption on the normality) and therefore be random normals themselves.  So effects on the qqnorm plot that deviate from normality are suggestive of significant effects.
Thining about it I asked a yes/no quesiton so this does actually answer it. Thanks.
Can you describe the data a little more? For example, is your respondent student grade per student or average grade per school? If it's the first I would use a mixed model of the following form GRADE ~ NEW_COMPUTERS + 1|SCHOOL in R-notation. You would still have to do the usual checks for roughly equal variance, normality,...
you can find a very nice and brief explanation on label bias here: http://liqiangguo.wordpress.com/2011/04/19/the-label-bias-problem/
@mp A glance at @Fan's [previous question](http://stats.stackexchange.com/q/10981) indicates this is not homework.
@Harvey - "exactly the same next week" also involves assumptions; assumptions of some kind are inescapable. $\\quad$ Kacu - Before you consider R, you need to consider what problem you're actually solving (in explicit terms, not merely 'predict energy consumption') -- you need some kind of model/assumptions. You can't just predict out of thin air, you need to know, infer - or at least assume - something about what is going on. When you say you have energy consumption for two weeks, is this weekly consumption? Daily consumption? Hourly consumption? What other information/context do you have?
@Adam I updated my response with the case of 2 categorical variables + a continuous response variable --  hope this is what you meant. I also added code to show how to customize `ggplot` theme. Generally, you can say `gp + theme_bw()` to just remove the grey background; here, I also removed the grid.
I believe you will find the answer here, http://stats.stackexchange.com/questions/109735/homoskedasticity-assumption-varyx-varux-constant/109748#109748, keeping in mind that $E[\\epsilon\\mid X]=0$
Use the inverse form of $W_p$ to get the invariant $\\Phi_p$ representation. Moments follow by the L-statistic decomposition, and then easy to check independence.
Thanks that would explain my result. I have one other concern though which I have written as an update to my original question.
The verbose notation gets in the way of the question itself.  (Why on earth do you need *two* subscripts $i$ almost everywhere?) Are you just asking how to generate iid Normal samples efficiently on an affine subset of $\\mathbb{R}^n$?
There is also a whole pool of methods for covariance matrix estimation. For example, described and implemented here http://scikit-learn.org/stable/modules/covariance.html
I found those responses to related questions ([What is the difference between generalized estimating equations and GLMM?](http://stats.stackexchange.com/a/17403/930), [When to use generalized estimating equations vs. mixed effects models?](http://stats.stackexchange.com/a/16415/930)) very comprehensive, although they are about GLM *with random effects* vs. GEE.
So when you arrive at a bus stop & someone tells you that you just missed a bus; & when you arrive, & someone tells you the last bus was half an hour ago: you'd estimate the same waiting time in each case? The memorylessness assumption seems rather strong for this case, even if you don't live in Switzerland.
I am a Data Scientist/Physicist so have never seen a line of R. As I suggested in the question, if it is a code issue you should ask it in Stack Overflow and you will get much more attention. However, the above answer is correct unless you would like to elaborate on how it generalizes to a median-unbiased estimator. For more details see page 172 of E.T. Jaynes book Probability theory ISBN 978-0-521-59271-0.
To me this is not obvious. I see clusters that in reality have different sizes all the time...
possible duplicate of [Simulation of Logistic Regression Power Analysis - Designed Experiments](http://stats.stackexchange.com/questions/35940/simulation-of-logistic-regression-power-analysis-designed-experiments)
You might find the following thread interesting / confounding: [Swapping X and Y in a regression that contains a grouping predictor](http://stats.stackexchange.com/questions/52645/).
Thanks. I think regression is the best option than logistic after obtaining factors. Do I have to use each factor separately as dependent variable in regression (not logistic)? or Can I have one factor (dependent variable) by averaging the obtained factors (or any other method)?
Won't there be a problem because the t test assumes the observations are iid, whereas the data in the OP's post will probably be autocorrelated?
Using classification accuracy to judge the performance of a model is discouraged in this situation for precisely this reason. A better alternative is the area under the ROC curve.
I start to have a feeling that even `glm()` is bayesian :-)
See the [Coupon Collector's problem](http://en.wikipedia.org/wiki/Coupon_collector%27s_problem) (and search on that term here or via google may turn up more useful information)
Is this a question of "How I plot the tree in R?" or "How do I visualize clustering, in general?" ?
Can you post the `cch` command you are using?
A substantive comment: when values are additive such that totals make (e.g.) physical sense, the mean is a a natural summary regardless of the distribution of the individual values.
@Ankur Unless you want to develop this quick reply, I have converted this to a comment for the moment.
(+1) Quite comprehensive!!
This is nothing to do with stats. The issue here is simple logic. I know you already know what probabilities are in your table, since you state it in your question. The first thing you must do is write down the quantities you need to compute. THEN worry about how to compute them. You know your numerator and denominator. Write yourself a simple question: "How do I compute P(X=8)?" and then think about it for more than a few seconds. I bet you already have the p.f., but in any case it's simple to compute it from the table -- you just can't look up P(X=8) in a table of cumulative probabilities.
For blending, this paper from the netflix competition is worth reading: http://www.the-ensemble.com/content/feature-weighted-linear-stacking.
Agreed! Although now we're getting into a different topic :). You could say the same about per-feature normalization.
If you have one group, and you are estimating its normal distribution, then your question applies to the form of the prior over its scale parameter. As my answer implies, you can use a t distribution over its log transformation or choose a fat tailed distribution with positive real support, being careful about other aspects of that distribution such as its kurtosis. Bottom line, if you wan a robust model for a scale parameter, use a t distribution over its log transform or some other fat tailed distribution.
Right! Thanks. And thanks for the edit.
This is explained in various contexts, such as the thread on sampling errors for polls at http://stats.stackexchange.com/questions/16413/.  You may also find that searching our site for [sampling error](http://stats.stackexchange.com/search?q=%22sampling+error%22) uncovers helpful explanations.
Hi, Thanks for the reply. The sample size is 76.
I am performing the Spearman's correlation because it is questionable to assume likert scales as interval scaled. Since spearman's rho is better at validating ordinal scales than pearson's, then I am using spearman as a supplement.
I am running a linear regression on these two variables, but the sig is 0,136 in the ANOVA, but that is because it is 2-tailed. 0,136/2 = 0,068 and the correlation analysis is made as a one tailed analysis, so that explains it. 
I do not know if I am too unclear, but you have helped a lot.
It's hard to do this without knowing the context of the analysis. What were the dependent and independent variables? What model was run? From the F, it looks like an ANOVA with 3 groups
Are you really comfortable with the practice of deciding that you have too much data because you don't get the answer you like, so you throw away data until you get the answer you want?  If this were good science then we could use this to prove that global warming (or other pet/hot topic) is real or nothing to worry about by just choosing the appropriate null hypothesis and throwing away data until we do not reject the null.
Following from the weight example - are the repeat measures at baseline (or followup) supposed to measure the same thing, with differences between measures reflecting only random error? Given baseline or followup, does the order of the repeat measures matter?  More detail in the question on exactly what you have measured and what question you are trying to answer would be helpful.
R code ported to javascript: http://bl.ocks.org/4108269 @Zen, your answer is better, my answer way just because I didn't have enough points to comment.
Any linear model maximizes correlation between a linear combination of predictors and a linear combination of predictands. So, MANOVA with DVs X1 X2 X3 and IV Y is an exact mirror of multiple regression of Y on X1 X2 X3. The quantity you call Pillai's trace in the former _is_ R-squared in the latter.
I'm sorry, I don't follow what you mean by 50,000 "significant dimensions" when each "consists of 10 values." I initially understood the latter to indicate you have data that are only *ten* dimensional. I'm not sure in what sense you are using "significant," either: that concept is relevant to hypothesis testing but not to PCA (at least not as it's usually employed) and the idea of using a 0 to 1 scale suggests your meaning of "significant" differs from what most readers on this site would assume.
@Tristan - Thanks. Interesting and certainly seems relevant.  Although, I'm not sure yet whether it answers my specific question--maybe I just need a better question, though.
I am still mystified: your description does not correspond to anything recognizable in standard statistical practice of any era.  I simply cannot tell what procedure you are attempting to contrast with bootstrapping.
I don't think we are arguing that Taleb thinks the normal distribution never makes sense.  It is just that for the examples he thinks are important he thinks it is improper to use it.  He is right about that but wrong in thinking that most statisticians do use it in those situations.
Is there Monte Carlo research or an analytic solution on which you based this answer?
Thanks for the response. This is made up data to simulate a T|F series totalling 100 dichotomous choices for each x point. So the limits are 0 correct or 100 correct but this particular case gets approx 15 correct. Using the betareg package... pacc <- accuracy/100 + 0.00001; b1 <- betareg(pacc ~ x) ... gives me the same looking regression as the binomial. Is this what you meant? Or are you suggesting imposing a post-hoc data based limit? In which case what distinguishes the beta from the binomial when both have been given post-hoc limits?
@mjburns In that case you probably have a similar scenario.  The model fits well over the given observation period but is not a "correct" model for the whole series. What commonly occurs is that the process changes.  Soemthing causes a nonstationary behavior to occur or a level shift.  This could possibly be due to an unknown intervention or a known intervention whose effect is unknown. Sometimes things happen that are unpredictable (like the black swan).  What you can do is revise your model as the new data becomes available and hope that the new model will then do better in your situation.
Do you want to test the null hypothesis that all the players have the same strength, or check the fit of a model of player strength?
To say that the outcome cannot be predicted with 100% accuracy based on available variables is not at all the same as saying one cannot model the outcome at all. @CodeAdmiral merely points out that prediction will be less than perfect. Now the task is to see how well one can predict using available data (that for all we know might relate to the outcome), given that individual competitors for an item will have unpredictable/unknown characteristics.
Using an indicator variable makes most sense when the difference between types implies an additive and/or multiplicative shift on the scale of the response. Even more important is that gesture type should be explanatory (causal?) in your scheme of ideas, not a response or outcome or effect.
Lucy, after your editing the question has become even more bloated and encyclopedic (for me). Each of your point might be discussed as a separate question... Please, define your priorities of interest.
I totally agree with your opinion about Gelman et al's book. Try *The Bayesian choice*, by C. Robert.
@Frank Harrell: Thanks for the advise. Actually, I first though that the problem arised from this log-log transform, and I tried to prevent SAS to compute any confidence interval but it seems that this cannot be done...
could you please be more specific what it is that you want to know and in what context? The way the question is currently framed is pretty broad.
Your argument implies the colleague is *not* correct.  BTW, squared residuals are not used to measure covariance: only the residuals themselves are.
Also `matplotlib` plots are ugly; they may be nice for a long-years gnuplot user.
Unless you have an extraordinary amount of data, it's possible there will be no solution to this problem at all, because estimating a one-per-million rate is quite an extrapolation. Absent millions of data, you would need a region of intermediate values of the predictor variables where varying proportions of passes are observed, separating a region of all passes from another of all fails.  You wouldn't be able to check for continued linearity of the response out in these regions, making the 95% confidence rather illusory.
I don't mean visual inspection, use a loop to detect which missing variables you have and select which one you'd like to use. It should be fairly easy to determine which combinations of factors have missing variables. R has some convenience functions for fitting nested models this way. Check out the help pages around "dropterm" and related functions.
Ahh, right, I didn't see the unbalanced nature of the design. That makes much more sense now.
I also don't know how to tag this because I do not know the right words; requesting tag improvement, please!
I doubt there is a formal method worked out for such a specific situation. You will probably have to use simulations. I don't know what software you use, but there are `R`-based examples of simulations for power-analysis on CV. I wrote one up here: [Simulation of logistic regression power analysis - designed experiments](http://stats.stackexchange.com/questions/35940//35994#35994).
I was connected to the internet since I managed to install the package :)
Another good thread to look at (besides the duplicate) is http://stats.stackexchange.com/questions/31/what-is-the-meaning-of-p-values-and-t-values-in-statistical-tests, which discusses the meaning of p-values.
@cardinal: Oops, thanks, I forgot to write that. Corrected.
How many conditions do you have? Unless you have like 10+, it probably shouldn't be a random effect (it should be a fixed effect). Are they a sample of conditions among a population of possible conditions? (Stated differently, do you want to generalize to these specific conditions, or to a larger population of conditions)
@user1600 Good point.
I think that You can write this as an answer. There is everything what I was looking for and I will be glad to accept such answer.
I've added some annotations (lines starting with #) in the code.  I imagine F# can do this - it can do the simpler case of logistic regression - but I think you would have a fair bit of work.  R is free and not too hard to learn.
While MO is not SE I'll let it stay, but please do not cross-post unless you lose hope in getting an answer on the original site.
Even for on-line learning, if past patterns are revisited, they should probably be shuffled. I carried out some experiments a few days ago and saw what I expected to see: backpropagation performed worse without shuffling, and PSO wasn't really affected (although I have to say no statistical significance tests were carried out... yet).
Are you just running a logistic regression?
I guess the reason for this default behavior might be that refitting the model can consume a lot of time time and memory, which is not what people expect when calling predict().
you should always be [weary](http://cscs.umich.edu/~crshalizi/weblog/491.html) of linear regression on log-log plots because the fluctuations at the tail matter a lot more... the post I link should have some R and Matlab code that might be useful.