tttjjj/elt_bench huggingface dataset:
@JDav It is not that the test statistic under the null hypothesis is right it is the issue of power. Any sample size can give you a test staistic with the right type 1 error if it is an exact test. If it is asymptotic then the significance level does depend a little on having the sample size large enough that the asymptotic null distribution is close to the true null. But power is the key that you may be missing. If you get an insignificant p-value it cpuld be due to a type II error. Keeping the type II error down requires an adequately large sample size.
Was just typing in basically the same answer. Here's another link: [ Arlot, S. & Celisse, A.: A survey of cross-validation procedures for model selection, Statist. Surv., 4, 40-79 (2010). DOI: 10.1214/09-SS054](http://dx.doi.org/10.1214/09-SS054)
You got it right, I wanted to know if I can use t test for my sample. Thanks!
That's helpful. I think I'll use that in my teaching in the future.
@ Rosh Rosh: Please check the answer as correct so that it will he helpful for other users.
Some points to clarify: 1. Is your dependent variable the same for both countries?  2. How did you pool both country data? 3. What type of the conclusions you are trying to make? My general observation is that if you pool both countries, you must assume that the same association applies in both of them. This usually is quite strong assumption. If it holds, then simply adding country specific variables to your pooled analysis might be enough.
Ai and Norton's paper has also been criticized by other work like Puhani (2012). The article and other potentially useful references are linked here:
http://stats.stackexchange.com/questions/89513/difference-in-differences-estimator-for-logistic-regressions/89516#89516
Sorry to dissent, but that book is deeply unreliable. I gave many details in a review in  _Biometrics_ 57: 1273-1274 (2001). Much better sources are the books by Anders Hald and Stephen Stigler.
Thanks, but questions need to be understandable on their own, so if you can, please describe what this means rather than just providing links.  Otherwise you may severely limit the potential audience for this question and reduce your chances of getting great answers.
Your code is not complete.
Thanks, but is there any benefit to using clusterfly rather than calling ggobi directly? The website only mentions clustering methods, which are interesting, but not my primary goal just yet. gcexplorer has less informative website, but looks like it is for visualizing data after it has already been split to clusters. I will give them a try once I get to that point, but not what I need right now.
Then you are pretty much done, invoke hierarchical clustering from sklearn.
@Kit: Writers? what would they know about statistics? (or here for that matter). This is a question about technical localized vocabulary. It is a great question for statistics.SE (where they 'own' such terminology)
Thanks for your clear answer Gavin. What makes spatial autocorrelation fundamentally different from any gradient not included in the model? Say your study area was on a sloping hill, and the species of interest preferred lower habitat to higher habitat. Failing to include the elevation in the model would leave a structure in the residuals, would it not? Is it simply that spatial autocorrelation is (or was) forgotten about or not considered? (P.S. perhaps this is a poor example as inclusion of lat,long would account for this effect too).
And for those that don't have closed forms, we could use the EM Algorithm. For example, consider the zero-inflated poisson moddel: http://stats.stackexchange.com/questions/32133/how-do-you-use-the-em-algorithm-to-calculate-mles-for-a-latent-variable-formulat
@kkp the convention is to use the smallest tree within one standard error of the best tree. The best tree is in row 8 (7 splits), but the tree in row 7 (6 splits) does effectively the same job and is simpler, hence the 1 standard error rule would select it. Note Frank's comment in his Answer. Almost certainly you want to be using one of the bagged/boosted trees or random forest algorithms.
Okay, I guess misinterpreted this comment: "Note that LA1 may be equal to LB1 or to LB2, and that LB1 may be equal to LA1 or to LA2."
You may want to read this thread: [Does an unbalanced sample matter when doing logistic regression?](http://stats.stackexchange.com/questions/6067/)
Almost all the answers to the question you reference will apply after making only the obvious changes.  The CDF does not require probabilities to be the same--not at all!
1) yes, the question is somewhat ambiguous; there are, as you mention, many definitions of 'optimal': via information criterion, cross validation, etc. Most of the heuristic approaches I have seen to the problem proceed by stepwise predictor addition/removal: single pass forward addition or subtraction, etc. However, Hosmer & Lemeshow make reference to this method (a variant of work by Lawless & Singhal), which somehow 'magically' selects predictors by a single computation of an MLR (modulo some other stuff). I am very curious about this method...
@Chowza: that is basically the idea - if you anyways run on a (random) subset to get runtime and memory measurements, you can also look how the results look. Of course extrapolating is very uncertain, but you can hopefully get a rough idea.
Your mention of the Jacobian-confuses me. The jacobian of the likelihood ratio?? $$\\frac{a_1+\\epsilon}{b_1+\\epsilon}-\\frac{a_2+\\epsilon}{b_2+\\epsilon}$$ is the difference where $\\epsilon\\sim f_\\epsilon(t)$ or with different epsilons. Also, feel free to take an example one-parameter distribution from the exponential family-say a poisson-if you would like to compute-and see if there is a caveat; or more details without any specific distributional assumptions is fine too. Also, considering a $\\chi^2$ might lead to a connection with an F-dist in the ratios, as a ratio of $\\chi^2s$ is an F-dist.
@Ger The line in the first plot is the logistic curve that best fits the data. The randomization is supposed to give you a sense of having equal density of data in each region (if the fit is good). After posting, I was thinking it might be good to overlay the composition of proportions with the logistic curve as a better indication of how the densities compare. Of course, the smooth densities are subject to a tuning parameter, like the binning in your originals. The examples are from JMP, a commercial product that I work on.
I cannot think of any situation in which molar can be confused with "millions". Millions is not generally used by itself, you usually write, for instance, 2M€, 2Mb or 2MΩ. Moreover, you use Molar next to a chemical name, es. CaCl2 0.5M, so there is no ambiguity.
I think @PeterEllis is right. The actual measurements are a physical parameter of a sample (fluorescence). The measured fluorescence is probably given by something like Fm = F0 + noise + drift(time), where F0 is the 'true' fluorescence signal, noise is random and drift(time) is some long-wavelength instrumental drift that is a function of time. I guess F0 and drift(time) must both introduce autocorrelation.
I see your point, thanks!
Maybe I don't quite understand the problem. But, consider a bivariate random vector $(X,Y)$ with the same marginal $F$ with full support on $\\mathbb R_{+}$ and having mean $0 < \\mu < \\infty$. How can such a bivariate distribution have correlation $\\rho$ close to -1, for example? Heuristically, though I haven't carried this out, it seems that if $\\mathbb P(X > 2 \\mu) > 0$, then a contradiction regarding the support must arise. No?
What does seasonal at the yearly level mean? Would equivalently say that there are week and year fixed effects? That there is a a long and short memory process going on?
It's good practice to flag that your code is R code, even if you think that's obvious. (Many users of other languages or environments think the same about their software.)
John, thank you for your answer, especially for the link to Gelman and Stern's paper.
I have used simple methods like sampling from joint PDF. Data maybe time series or gridded data. I am not familiar with the Netflix approach yet and have only read some preliminary details. Any chances of it being used to generate ensembles that reflects the structure of the true observed data.
Do the sample sizes reflect population sizes?
@kjetilbhalvorsen did you mean to post the same link in your second comment? The first comment briefly stated that one might construct a VIF using McFadden's pseudo-$R^{2}$, but there's not really any discussion about why that is or is not important to do this. For example, introductory textbooks on regression will stress the importance of checking for collinearity (e.g. by using VIF) in a multiple linear regression context, but ignore the issue when presenting logistic regression. Trying to understand why.
Answered at http://stats.stackexchange.com/a/18448.
Welcome to the site, @user2317915. It's not clear that this is an answer to the OP's question. Can you expand it into an answer (eg, by explaining how bootstrapping might be a viable solution)?
(-1) I am sorry for the downvote, but this answer is currently the top one in this thread and I am afraid it can be misleading. As others replies have already pointed out, in high dimensions LDA tends to overfit *badly*, because its computations rely on inversion of the within-class covariance matrix that is very difficult to estimate reliably unless $N>>k$. [See my illustrations here](http://stats.stackexchange.com/questions/106121). Of course one can use a regularized version of it, but "vanilla" LDA does not use any regularization, whereas SVM always does.
Thanks, @Glen_b. Upon looking back, I can see how unclear my question is. The question is how to combine information from Lab A and Lab B such that they are on the same scale. I've gone back to edit my original post, so hopefully that clears things up.
Well D itself is a another equation that is calculated by *other* variables: $D = \\frac{n_1}{2} * \\frac{N_2}{n_2}$. Does this still apply?
Could you mind editing the answer and adding the format of newdata. Because newdata doesn't have class label, how to match columns
Thanks Peter Flom,

The contingency table was my first lead. And the Chi2 test is highly significant indeed. But I'm looking also for a way to plot the result. I tried the image() function in R, but I'm not really happy with it, I would like to be able to plot it on axes (factorial maybe) and see which stores are closed one to another in terms of patronage.

For the network approach, I managed to get a plot, but it's too crowded to be insightfull. I will try the clustering methods of SNA, may be a good way to go.
(+1) Thanks. A very interesting answer.
Imagine two jails with the same number of cells and same arrival rate (we can even say it's equal to the departure rate, so occupancy stays constant). One jail has all prisoners stay for a week. Another jail has 90% of the prisoners released same day (zero nights) and 10% stay for 10 weeks. Same arrival rate, same occupancy rate, different distribution of stay duration.
I've used ArnetMiner! It went offline for awhile, but I'm glad to see it is back again. We talked about it on a statistics blog about a year ago. I'll try to find the URL.
How does that work?  Doesn't the GLM estimate *expectations*, which are not necessarily integral?
If $np \\geq 10$ and $n(1-p) \\geq 10$ then the normal distribution is a good approximation to the binomial.
@Michael, yes of course I knew what you meant to write but subtle errors can sometimes be misleading for novices - thanks for correcting.
Thank you. For the AIC value is it some acceptable value? for example a model's with AIC 1277 and another with AIC 1067. Can I say that the second model is better? or the AIC is still to high for making that assumption?
Possible duplicate [here](http://stats.stackexchange.com/questions/12398/how-to-interpret-f-and-p-value-in-anova)? --  and perhaps some relevant insight [here](http://stats.stackexchange.com/questions/50727/f-statistic-f-critical-value-and-p-value)
It depends on what the model is *for*. If the point is to predict, you'd care more about predictive accuracy. If the point is something else, you may not care about slightly better predictive performance.
many thanks for your advice.
I have tried the formula you mentioned using R. Unfortunately, the data set I used did not justify the formula. The lme() function applied simply did not converge to a numerical solution. Seems that the lmer() function in the lme4 package does the right thing.
I could not make any assumptions on how sensor produced (x,s) values. They might be based on some hidden iterations or perhaps something else. However it is safe to assume that each (x,s) pair returned to me is independent from others and measuring the same **true** value of 'x'.
@ttnphns, My understanding was that the OP just wanted to know how to get the PSM output.
Not sure I understand your description, you have 16 items but five components? Have you got four or five motivation categories? Incidentally, how many respondents do you have in total?
This is the so-called "Shorth" of the data. For perfectly symmetric distributions, it corresponds to the box of the boxplot.
Thank you ttnphns. I saw your answer before. But that has no reference on how the steps are derived and thus not what I was looking for. Moreover this SPSS output or whatever the syntax is is not very readable. Thank you anyway. Due to the lack of interest in this questions I'll look for the reference and use another estimate for the variance.
+1 for convolutional neural networks. :D
Why does it not test the hypothesis "a HCA exists"? It tests whether there is an imbalance towards home wins which is significant - meaning assuming a random model with no imbalance will yield the observed result or extremer with probability lower than a chosen level - which leads to acceptance or rejection of the null-hypo ... this seems pretty straightforward to me.
I don't see anything that establishes the y's have mean 0
Can you please clarify what you mean by "how can i cut some factors of my 22 factors by mean value"? Do you want to reduce your 5-point scores to binary indicators?
(1) Is this for some kind of study?  (2) Why not post your empirical answer first?
Ah ha. You've ignored the centering that `prcomp` does when you've computed your coordinates. Try adding `center=FALSE` to your `prcomp` call and then repeating the segment plot.
@DilipSarwate: (1) An ARMA process isn't necessarily causal, because the solution to (3.1.1) is (3.1.3) which shows that an ARMA process may not be causal. (2) A causal ARMA process is defined in (3.1.5) in the link in my last comment, where the definition of an ARMA process is still (3.1.1). Do I misunderstand something?
2) is the right model.  That was as far as I got as well, but my application in particular (a particle filter) relies on the ability to weight each particle according to the "probability" of an observation.  If observations are all discrete I can use the Prob. Mass Func. or if they're all continuous I can use the Prob. Dens. Func., but what do I do when they're mixed?
I am not modelling stateful data, rather interaction (events/moments) data. The choice of $\\log(q)$ and $\\log(1-q)$ just enables the past log odds to be taken into account directly. It is the same logistic transformation applied to past interactions to predict future interactions. What is the alternative in formulae?
Simple: it's "grading on a curve."  :-)  I give an algorithm at http://www.quantdec.com/envstats/notes/class_03/probability.htm ("Reading Q-Q plots").
@dashi: yes, double cross validation is similar to your method 2. But at the outer level it does not only split once, but does a full resampling validation again - as you correctly say. I added a literature reference.
Thanks gung, this answer is great on a couple of levels. Firstly it answers the main point I was interested in. Secondly, you predicted what my follow up question would be, because I was going to ask how this changed with the introduction of interaction terms. Thanks for the math as well. I know this question is kind of basic but I feel that you can never be too explicit with these concepts.
Thank you, @Avraham, for trying to bring this unanswered thread to some closure by posting an answer here.
If you compute all the expected values $E_i$ in terms of the observations $O_i$ then you have no degrees of freedom left. This tells you two things: (1) If your value of $\\chi^2$ is not zero, then you have made a mistake and (2) you don't have a test at all.
I've edited the original comment to describe the ESD test.
Ah, well, reading about those is actually my homework for Thursday \\o/.
Sounds very risky.  I recommend [Cosma Shalizi's notes on factor analysis](http://www.stat.cmu.edu/~cshalizi/402/lectures/18-factors/lecture-18.pdf).
(+1) I especially like the last paragraph.
If the probabilities are not too close to 0 or 1 the least squares fit might be OK, but otherwise it is not well approximated an unbounded continuous variable, which least squares assumes. What software are you using?
@ttnphns, I am afraid my previous comment was not delivered (I forgot to put your user name), so let me repeat: Wow, thanks a lot, your linked answers seem to be very related to my question and I must have missed them in my search before posting. It will take me some time to digest them and I might come back to you after that, but maybe you could already now point me to some papers/books covering these topics? I would *love* to see a detailed discussion of this stuff in the style of your linked answers.
I also ran the ANOVA using a square root transformed version of the data, and now some of the variables are significant that weren't before. It's left me more confused about what to do!
I'd suggest a hypothesis test with the null hypothesis that 50% of the students in the underlying population will choose the first answer. Search the web for "hypothesis test proportion". Also see answers on this site with the [hypothesis-testing and proportion tags](http://stats.stackexchange.com/questions/tagged/hypothesis-testing+proportion). E.g., http://stats.stackexchange.com/questions/84209/statistical-test-for-testing-equal-proportions.
Sometimes weird stuff (even statistically significantly weird stuff) happens. Indeed, if p = .05 exactly, stuff that weird will happen 5 times in 100. That's not rare. 

Instead of focusing on p-values and whether they changed in various ways, I'd suggest focusing on effect size. With your hug sample, almost any effect size will be significant. However, it may not be of any practical importance.
Thank you for your answer, appropriate axes was my problem as well, I am very new to this, so I have no idea. I already have the top 10 words that I would like to use as my labels, I am just using the top 15 most frequent word in every cluster (I said 10 in the post, will change it now), it is the last for(1 in 1:k) that does that, but I just used that to see what words it turns into clusters, to check whether it makes sense.

What I would like to accomplish with the visualization is to see at a glance the difference in numbers between clusters and how similar these clusters are to each other.
Are you using SPSS? Can it not just be that 7 out of 8 independent variables are found not significant and therefore dropped out of the model?
But regarding the most important independent variable I agree more with @Ben Bolker
.Now is it the correct conclusion (t-test suggest me variable 4and 5 are significant but according to CIs only variable 5 is important)?
Stepwise regression without simultaneously incorporating shrinkage is an invalid technique.
**Related**: http://stats.stackexchange.com/questions/5960/how-to-identify-a-bimodal-distribution
Same as your answer [here](http://stats.stackexchange.com/questions/50770/is-anova-being-misused-for-these-binary-data/112742#112742).
I have no idea. You are going to have to provide more details for this to be answerable. Eg, what analysis do you intend to conduct?
+1, this is a really good answer & there is a lot of info here, but 1 suggestion: you make a subtle point in your first sentence, but then move on to more concrete issues. It might be worthwhile to have a little semi-philosophical explication of what it means that "there is not a right answer here".
Typically 'bubble graph' is on an x-y coordinate system with no connections between bubbles.
Michael, you wrote your answer in your comments (I suspect that you did it by mistake). And now that you mention it, yes, maybe the author meant to express $P(-z<Z<z)=\\alpha$ as $P(0<Z<z)=\\alpha/2$!
I think you've got it. There's a difference between 50 different people, and 50 copies of the same person!
I'd also add, these days I tend to use kernel ridge regression rather than SVMs as you don't have these sort of counter-intuitive problems due to the discontinuity in the derivative of the loss function.  Quite often if you tune an L2 SVM properly, you end up with a very small value of C and all the data are SVs, at which point you have a KRR model anyway.  The more I used them, the less useful I have found SVMs in practice, although the theoretical insights they have brought have been vital.
excellent answer, thank you!
what is the name of this paper ?
I would like to suggest that making a consistent distinction between *estimates* of skewness/kurtosis and their *population values* would clarify this question (and perhaps even answer it for you). It seems clear that this distinction completely resolves question (1), because missingness has no effect on the population moments. What I cannot figure out from your text is whether you are specifying skewness/kurtosis/correlation for the *population* or for the *sample*.  (And if it's the sample, then *why*?)
You've confirmed my instinct. Thanks, Mark!
If you don't know the operational class frequencies, they can be estimated by EM without knowing the labels of the test/operational samples.  The details are in Saerens et al. "Adjusting the Outputs of a Classifier to New a Priori Probabilities: A Simple Procedure", Neural Computation, vol. 14, no. 1, pp. 21-41, 2002 ( http://dx.doi.org/10.1162/089976602753284446 ).  I've used this a couple of times and was impressed at how well it worked.  Note however that the theoretical correction is not normally optimal, and setting it via e.g. cross-validation is often better.
@conjugateprior Check [this out](http://www.uri.edu/research/cprc/Publications/PDFs/ByTitle/Dynamic%20Factor%20analysis%20of%20non%20stationary%20multivariate%20time%20series.txt.pdf)
I see, thank you for your help!
Hint: By [definition](http://en.wikipedia.org/wiki/Simple_function), $e$ is a finite linear combination of indicator functions. You can explicitly construct the $A_i$ from the supports of those indicator functions. If the procedure is not obvious, do an example with $m=2$.  BTW, what does "$\\varepsilon$" stand for?  Do we even need to know?
+1 Great suggestion.
Thank you. I overlooked it.
$\\cdot$ I'm interested on what you said about residus? How could I learn something from them. I have already ploted them and indeed seen strange stuff such as a mean different from 0. How can I study residus in general?
$\\newline$
$\\cdot$I see what you mean by "overfitting": The model that gaves me $R_{adjusted}^2=0.87$ was actually built from another set of data for which it gave $R_{adjusted}^2=0,95$.
Under what software/algorithm is your model not converging?
+1  This is fantastic. Thank you :)
I'm reluctant to recommend EFA without knowing what kind of data we are dealing with: introducing a model for the errors (which PCA doesn't) has certainly its advantage when dealing with targeted latent variables, or more generally when trying to uncover latent structures, but PCA (with its caveats) is mostly used to perform dimension reduction, or feature selection in large dimension, AFAICT. In the $n\\ll p$ case, EFA would be inappropriate while sophisticated methods for variable selection do exist. I don't know of the OP's case, so I cannot say more, but this is a good remark.
and also the first post to this series http://lpenz.github.com/articles/df0pred-1/index.html