tttjjj/elt_bench huggingface dataset:
"On contrary, if two values are close but not identical, I'd consider it much more likely that there is some slight error in one of the two implementations." I disagree. If you assume a uniformly spread dataset, it is possible that the the centroids are not close but the WCSS are close.
"But in the end, k-means is so simple, and such a crude heuristic, what good is it to compare two results?" - Do you mean to say that verifying k-means is meaningless. I mean, I can look at 2 different values of total WCSS and come to a conclusion that one way of clustering is better than the other.
@Glen_b - I take your point, but there's nothing explicitly in the question that says this is a pure learning exercise.  If it is a "real" problem, then mcmc would be crazy for a two parameter problem.  MCMC should always be the method of last resort imo.
@GuessGucci, Not "loadings". Loadings are the coefficients to predict an original variable by component(s), To predict vice versa, use eigenvector elements as the coefficients. Also, you should initially center all your variables, because PCA is a regressional model without intercept.
Also, the data seems to be missing completely at random. I believe it's a quirk of the experiment presentation software (psychopy) - every few hundred trials or so I get a null or negative reaction time.
@AaronMcDaid: Good idea, [done](http://stats.stackexchange.com/questions/56086/most-efficient-way-of-sampling-a-product-of-distributions-k-cdot-fx-cdot-g).
ANOVA applies to different sized groups. It's sometimes called 'unbalanced ANOVA'. The balanced case has some nice properties but it deals just fine with the unequal-sample-size case. There may be some other issues to consider, but unequal sample sizes isn't an issue.
Also, see below for a second answer, which may answer exactly what you need. However, I recommend for clarity that you edit your question, so it can be more useful to the community.
For more on the single equation generalized ECM, see Banerjee, A., Dolado, J. J., Galbraith, J. W., and Hendry, D. F. (1993). *Co-integration, error correction, and the econometric analysis of non-stationary data*. Oxford University Press, USA. , or De Boef, S. and Keele, L. (2008). Taking time seriously. *American Journal of Political Science*, 52(1):184–200.
What is the bigger problem you're trying to solve? Why can't you use $m = n$?
+1 This way of thinking has an honorable history: it was used (quite effectively) in [regression analyses as early as 1755](http://stats.stackexchange.com/questions/46019/why-squared-residuals-instead-of-absolute-residuals-in-ols-estimation#comment89550_46019).
Hi Mike, Thanks for your help. The actual independent variable that I am measuring is the magnitude of an evoked response measured with EEG. I was trying to make it simple to avoid extra explanations about the data. I am also using a specific set of stimuli that I am including in the analysis (a unique value for each stimulus) as a within subject factor. I think this should be OK.
More information will be required to answer this question.  Specifically, how variable are the scores within each of the groups and/or what kinds of numbers are these?  Normally distributed continuous numbers?  % correct?
Thank you @chl for enhancing my answer with two well-chosen links.
Perfect, thank you! I didn't think to use plyr.
Good advice. I just want to suggest that some of it is context-dependent; for instance, non-normality of residuals can have critical effects on *prediction limits*. Thus there can be merit in achieving residual distributions that are not wildly skewed, at least.
By E(X) I denote the expected value of X, i.e. population mean. Does that answer your question? Also, you are right in assuming that the quantities are positive. To give a bit of context, my friend made it sound like each random variable (X, Y, W, Z) may be the weight of (distinct) mice exposed to different experimental conditions.
Thanks for the answer -- I'll give it a close read.
Note $x\\log{x}$ tends to zero as $x$ does.
@charles.y.zheng I knew that ;). If you think that such a probability cannot be calculated by frequentists, why not submitting it as answer. If the community agrees, I am happy to accept it :).
+1 for omegahat - very good source
"The prime example is fixed and random effects.  Wikipedia articles about these terms are a mess, mixing econometrics with statistics." So true.
Elegance maybe - but cost?  I think companies care about that!
Oh, and user11852's answer in regards to AIC with Gavin's addendum stands, there is no special AIC.lme or anything to address that issue and the whole thing starts to slip beyond my pay grade
Thanks for this, I totally agree with the stoping if the experiment is inevitable, that really makes sense! With that 'stopping if its really unlikely' I would doubt a real frequentist would agree. It's nothing less then saying: Hey, 95%? I would say 93% is also good! I mean it would be also good to just be satisfied with 90% confidence, but as a frequentist before looking to the data!
@jbowman your formula for $\\rho$ looks consistent with the OP's (aside from the inclusion of the factor $c=1$), but (unless I made an error) it looks to me like you put in the service time for the service rate.
You can change the reference level for your factor with `cloneRe <- relevel(clone, ref="A21")`. `aov(mu ~ cloneRe)` should then compare all groups against level `A21`.
i just mean that XZW is a very particular interaction, why not use Xlog(Z)tan(W) or product of other transformations.  but for discrete variables X,Z,W say in (0,1), there are exactly 8 combinations and interaction XZW means exactly one of them (1,1,1), another form, say (1-X)Z(1-W), will point to another combination (0,1,0)
Where the Hessian is available, the estimate of the covariance matrix is usually obtained as the inverse of the Hessian. [With Fisher scoring, I think there's an alternative approach that can be used.]
Pete Dixon wrote an article back in 1998 called "Why scientists value p-values" (http://www.psychonomic.org/backissues/1631/R382.pdf) that might be an informative read. A good follow-up would be Glover & Dixon's 2004 paper on the likelihood ratio as a replacement metric (http://pbr.psychonomic-journals.org/content/11/5/791.full.pdf).
mjc, thanks so much for the comment - this is exactly what I was looking for.
I added a couple of sentences to my answer, one just below each of the two diagrams.
Bootstrapping can both estimate and correct for bias, though samples need to be fairly large.
Well, that solves the problem with the intercept but the coefficients are still wrong. The missing intercept also was just one reason why I think I'm using the wrong model. Apart from that, my results are just plain wrong.
Thank you for this example. It helps a lot fix some ideas.
I've been using the `sp`, `spatstat` and `ggmap` R packages for a much smaller project and they've worked well. My concern would be how well Google covers the areas of India you want to geocode.
what kind of restrictions you're looking for? i dont think there's an answer to a general restriction
2/2 From looking at that distribution, you can see what time frame had the most number of visitors by seeing where the largest peak is of the histogram. And by zooming in, you can see the distribution of busiest time period (say from time A to time B). None of that changes the fact that the distribution is typically Poisson. So again, I don't think you understand what you mean by the "busiest" time period.
Thanks! it's the kind of book that I search!
@Gael That's good to know! I wish you nice coding sprints :)
Good points! @user11852.  I was thinking about response surface methodology as part of my answer but left it off because maybe it is more useful for designing experiments than searching for global extremes of a function.  But the idea of a locally quadratic function to search for extremes might make some sense. But as you say with noisy data and no real idea of a functional form it can look like a function with several local minima and search algorithms can trap you at a local minimum.  Simulated Annealing is certianly one approach that can pull you out of a local minimum.
Have you tried simulating some data to see how it behaves?
Are you allowed to keep tossing after you hit \\$0? And if yes, and you get heads, will you go into debt or stay at \\$0? This will significantly affect the answer.
@whuber I believe that the post is referring to the multiple comparisons problem.
@Algo: it does clarify the original question, but of course a diverse audience will make it very difficult to find common themes. For one thing, physics and math majors will have a math background very different from an English major; on the other hand, I found natural science and engineering resist the very idea of uncertainty, and have hard time conceptualizing it. You might find something useful at http://www.math.smith.edu/natscistats.php and/or http://www.causeweb.org/.
+1 Thank you for providing an exemplary answer that shows this thread *can* be useful and provides a standard of exposition that other replies can (and should) aim for.
Welcome to the site, @Abolfazi. I wonder if you're answering the question 'when is quantile regression *better* than OLS?'
It is a well-known fact that a non-significant omnibus test does *NOT* imply that none of the comparisons subsumed under that omnibus test are significant. This has nothing to do with `aov()` vs. `lme()`.
What do the x and y axes in the plot mean?
The newer version of `lme4` (version 1.1-7) has different warning behavior that the authors believe is less likely to give "false alarm" warnings. You might try updating `lme4` to the newest version, fitting the model again, and seeing if you still get similar warnings, at least in the first case.
@Thylacoleo This is a very crude basic and greedy method. Often you keep your validation set the same over runs, but whatever you like is ok.
Previously asked at http://math.stackexchange.com/q/31871/18398
$\\hat R$, the potential scale reduction factor, is essentially a scaled version of the estimated variance of the stationary distribution.  The variance is estimated as a weighted function of the within and between chain variances. Approximate convergence is reached when $\\hat R \\approx 1$.  Andrew Gelman developed this and a google search will come up with plenty of details.
Thank you for your answer. I'm using classification accuracy as cost function of my optimization algorithm. My model is searching to find best structure of neural network (inputs and number of neurons) based on classification accuracy + I'm using `5-fold cross validation` and this accuracy is on the test samples. With these information have i same problem? Can you describe more your suggestions? Thank you so much for your helps. I think random forest is a structure for decision trees and I'm not familiar with second suggestion.
Andrew, because the statements in your first two paragraphs appear to contradict the Weak Law of Large Numbers, please provide demonstrations of your claims or references that we can check.
Thank you for the alternative way to express C and the extra info
@oadams: it's pretty difficult to do manually. You have to fit a normal model to the data first, and then figure out the probability of each data point according to that model (or probabilities of quantiles in the data and the model).
Thanks I got it. So they are not independent.
No I haven't used any explanatory variables. I have juts given the input for independent variable. Secondly, I am using Expert Modeler option in SPSS which has given me the ARIMA(0,0,1)(0,0,0) as the best model..
Adding on to what @amoeba said: you should also consider *upvoting* answers that you like (whether or not you accept them, whether or not you asked the original question) by clicking the up arrow to the top left of the answer.
(+1) And not every case that can influence (distort) coefficients is an outlier.
So the python code the OP posted is incorrect? Or at least: sub-optimal? Thanks for the link! I will most probably base my handling of RBMs on that guide.
that is to say, that naive Bayes is the preferred choice with limited number of training example...
For those looking to this answer and the comment for help, note that nomad545 has also made use of the `arm` R package for the `se.ranef()` function.
Ok so I will report all parameters and won't exclude them at the final model. Thank you Frank Harrell.
Your answer is quite confusing. I'm not sure I understand what you're saying. Anyhow, my question was *why similar cross-entropy error have different accuracy for training and testing set*. Accuracy, or better, #correct guesses over #total samples is a pretty solid figure, i.e. reaching 100% tells me I can predict the whole testing set labels given its images. Perhaps you have answered my question, but I haven't got it. Could you please reword it?
Can you try to determine when the second derivative is 0 or close to 0?
The conceptual issue I had is that if you are comparing enough models, you are effectively fitting on the validation data when you "decide on a winner" using the validation data.  Hence still may be a point in partitioning the validation data.
(+1) Nice response.
+1 for modelling unpredictable deviations as random variables, the tickets from a box analogy and critical region references. I've updated my question with a more concrete (though hypothetical) example of what I'm interested in. I'll check out those books. I was hoping a widely accepted method exists for flagging significant results in a set of data based on prior data.
@ Glen_b The main reason I do this is because I read that I have to check those values (Cook index and so on) and if those values are too high it is recommended to exclude them. I really don't have much clue about statistics so I just follow the instructions in my book.
Thanks for the insight... Especially the point about balancing the samples between genders. That thought hadn't occurred to me.
If both variables are dichotomous, Pearson = Spearman = Kendall's tau. Yes it may have sence. With truly binary (boolean) data it also make sence to compute "Pearson" on data without centering, that would be cosine.
Do you get bonus points for the first graph in the stats exchange?
@fRed I have machine learning algorithm, but I'm not sure about its reliability. No real money, this is an academic research.
The normal-Wishart choice is indeed the default choice for mixtures as it gives you closed-form formulas and a straightforward Gibbs sampler. There is no non-informative version as you need the prior to be proper. For the choices of hyperparameters, I would pick $m$ as a central value of the dataset, $\\Psi$ associated with the scale of the dataset, $n_0=1/n$ and $\\nu$ equal to $3$ to $5$ to ensure existing moments.
Many questions: 1. Is it an experiment or an observational study? 2. State clearly what are your predictors and what is your outcome variable. 3. Try to be more specific. 4. What do you mean by sorting? 5. What is your sample? Is it n=100 pieces?
@Glen_b thank you very much for all your time and effort!
Histograms are poor tools for estimating distributions: see http://stats.stackexchange.com/a/51753. Therefore it would be foolhardy to hazard a guess based on this one. Certain forms of summary statistics provide more powerful tools for estimating distributions. In addition to the usual (first few moments, etc.), having extreme quantiles (the quartiles, eighths, and so on) can be very helpful.  The [first four moments](http://en.wikipedia.org/wiki/Pearson_distribution) alone often do a good job.
So, you can't use any cases (observations, records) with missing values. Your software should take care of that somehow. (Depending on quite what your scale is, correlation might not be best, but that's another story.)
Under the Bayesian approach it isn't true that there is actually a 95% probability that the true value lies in a 95% credible interval.  It would be more correct to say that given a particular prior distribution for the value of the statistic (representing our initial state of knowledge) then having observed the data we have a posterior distribution representing out updated state of knowledge, which gives us an interval where we are 95% sure that the true value lies.  This will only be accurate if our prior is accurate (and other assumptions such as the form of the likelihood).
@lokheart `anova(model1, model0)` will work too.
Thanks. That's what I started with. But was using an old edition of Walpole which did not have much information on it.
he is using Java ... so for the truncated Normals, use the inverse CDF method. This is the corresponding R-Code, should be easy to adapt in Java:     FA = pnorm(((a - mu)/sigma))
    FB = pnorm(((b - mu)/sigma))
    return(mu + sigma * qnorm(runif(length(mu)) * (FB - FA) + 
        FA))
Hello, Yongfeng, and welcome to the site!  I have to say, though, that your question really looks more like something specific to the java API of Weka, rather than a statistics question.  There is a weka tag in stackoverflow, so http://stackoverflow.com/questions/tagged/weka may be your better chance of getting a rapid answer.
This is basically a duplicate of http://stats.stackexchange.com/questions/82720/confidence-interval-around-binomial-estimate-of-0-or-1/82724 (that question has an additional assumption of a small sample size, but the answer is relevant no matter what sample size is used). One important point is that you shouldn’t use the normal approximation (the default in many software programs) *even if* the observed proportion was slightly higher than 0%; it has truly terrible properties for all probabilities close to 0% or 100%.
@Geoffrey, whatever the autocorrelations, they're benign. I thought you ran the ARIMA model. maybe I'm confused. If you ran OLS, and got these nice residuals, then i'd stick to OLS, as long as your coefficients look good.
Following up on my last comment, I tried the following, which I think accomplishes what @probabilityislogic suggested:
probably either an information-theoretic approach (AIC, BIC) or [Vuong's test](http://www.jstor.org/stable/1912557)
I'm not an expert in the topic, but I think some more details about the nature of your data could be necessary to peek proper way of imputation.
I don't have a number for their works. I'll look at your other suggestions.
So, if I understand you correctly... Does heteroskedasticity matter if you have a large enough sample? Yes, it does.  Does a large enough sample mitigates the heteroskedasticity issue? No, it does not.  And, calculating robust-Standar Errors and other methods like WLS, etc... works well to correct the heteroskedasticity.  Did I understand your answer correctly?
Computational poetry (PDF): http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.27.4337&rep=rep1&type=pdf
To make your answer more useful for novice readers, could you perhaps give a little discussion or explanation for your code?
Don't confuse the pretty label locations for the data. The figure region has been set to the range of the data. If you want axes with labels at specific, predefined points, turn off axis drawing in the `plot()` call with `axes = FALSE` and then draw the axes using `axis(side = 2)` for the y axis with suitable `at`, and the x-axis can be drawn using `axis.Date()` as per my answer below.
This approach is standard but it also seems pointless, because it's tantamount to putting a single observation into its own group and separately fitting it.  You might just as well remove the first observation and fit the rest of the data.  Note, too, that p-values, predictions, etc. will be problematic due to the interaction between model specification and model fitting that is going on here.
+1 Good, thoughtful discussion with a good recommendation at the end.  For more about the bootstrap/resampling/permutation version of the t-test, please see a recent thread at http://stats.stackexchange.com/q/24911.
What do you mean by "r<0.1"? Is this a typo (ie, *p<.1*)? Is it a correlation coefficient? Is it logistic regression coefficient? What tools have you used to try to answer this question? In #2, did you simple compare the probabilities (eg, .9 is 10x .09)?
Usually, we would consider {1, 2} vs. {3, 4, 5} (or even {1, 2, 3} vs. {4, 5}), but we would not exclude the middle category. That's if you really want to dichotomize an ordered categorical variable, which does not always makes sense, and do not want to use ordinal logistic regression or item response models.
+1 But please note that `subset` should only be used in the global environment. See also the warning in the help file.
It appears AIC is more for comparing models, and not for figuring out how well (in strict terms) your model is doing.  That would be more like RMS, right?
This is too broad to for a good question; you can collect tons of relevant information by [browsing our books tag](http://stats.stackexchange.com/questions/tagged/books).
You would be right if it were a cabinet conversation between you and the OP. But this site is being seen by many frequenters and passers-by who would be bewildered by an interrupted response. Once again: if you want to stop at a hint level, please do it as a comment, not an answer.
No, you would be undoing all the advantages of svm to do pre-svm feature reduction, at least if such reduction is unblinded to $Y$.
Yes. The data is a counting of 6 cell cultures where each have been treated with a different drug (6 drugs in total). For each counting the cells were divied into two classes: phenotype + and phenotype - (p+ and p-). So one counting can for example have 60% p+ cells and 40% p- phenotype. My question is if I can use a statistical test to produce some P-value for the increase or decrease of the P+ or P- phenotype after drug treatment compared to Control. The problem is I only have one replicate so I can not use the tests i normally use (ANOVA for instance).
You have a nice fast machine -- took my computer 2x as long to fit that probit model!
(here you go)[http://www.ece.uvic.ca/~bctill/papers/mocap/Aitkin_1997.pdf]  personally, while I do find it interesting, I struggle with the question of why the posterior distribution of the likelihood ratio is the quantity of interest.
thanks, your answer brought me on the right track!
It is curious that in your last line you link to my answer which demonstrates that the quotation is wrong! The quotation comes from a [different answer](http://stats.stackexchange.com/a/14525). (The vote counts might mean something.) Incidentally, multicollinearity is more general than your characterization in the bulleted item. For instance, in three dimensions you can place three collinear vectors at 120 degree angles from each other: they only have to lie in a common plane. Because 120 and 180-120=60 degrees are so large, no two of those vectors point in anywhere near the same directions.
What makes you assert that the AIC is high? I already explained to you [on a previous question](http://stats.stackexchange.com/questions/88083/interpreting-scatter-plots) that that residual deviance is *not* high. Do you think the answer will be different on different days?
(+1) Indeed, growth curve and mixture LV models are among some of the 'hot' topics in SEM or psychometrics; they are covered in some recent books, like *Latent Variable Mixture Models* (Hancock & Samuelsen, 2008). I have other papers on my TOBEREADFORTOOLONG list, and I would recommend looking at work from Múthen and coll., in conjunction with what the Mplus software offers for that [particular purpose](http://www.statmodel.com/discussion/messages/11/143.html?1327290364). If I find some time to reread the literature and compare `lavaan`/Mx to Mplus, I will post a reply myself.
I offer this answer as evidence that the thread should stay here and not be migrated as being solely of interest to `R` programmers.
I don't think that you're supposed to average imputed values.
Thanks, Maarten!
"*I am bothered with this huge difference in sample sizes*" -- why is this bothersome to you? (If you feel a randomization test is okay, why is a randomization test on the ranks suddenly a problem?)