tttjjj/elt_bench huggingface dataset:
You have framed this as an estimation problem (find the best) but are suggesting you want to do hypothesis testing.  In the absence of a prior distribution, the best guess you can make of the level that is best is the one that appears the best in testing: it's that simple.  In other words, choose the ad with the highest click-through rate.
I think you don't even need CDF of the bivariate normal distribution. You can determine the mean and variance of the distribution $Z: X+2Y$ and then use the CDF of a univariate normal distribution.
ANOVA does not have to normal data; it has to have normally distributed errors and I see no reason that a linear model fit to these data would necessarily violate that assumption (you'd have to run the model to see). If there is no censorship, then there is no need for survival models. It isn't clear from the post whether there is or isn't censorship.
+1 for the first few lines of your answer.  In my opinion, the reason to choose a Jeffreys' prior over a "non-informative" prior is not simply as a follower of Jeffreys.  It's because it really is like making no assumption whereas a so-called non-informative prior is making an assumption about the parametrization.
As a sidenote, I had the opposite effect with dropout on MNIST. Empirical loss stayed constant at ~150 errors, but the NLL went up again. Weird.
Welcome to the site, @btel. There is no need to sign your posts, in fact, we'd prefer you don't. Your avatar is automatically added to every post, along with your user name hyperlinked to your userpage, & other info. Since you're new here, you may want to read our [about page](http://stats.stackexchange.com/about) & our [FAQ](http://stats.stackexchange.com/faq), which cover info like this.
(+1) About R packages and other way to assess cluster stability, I provided some information on this thread: [How to tell if data is “clustered” enough for clustering algorithms to produce meaningful results?](http://stats.stackexchange.com/q/11691/930). And there are some [BIRCH implementation](http://stats.stackexchange.com/q/10059/930) available. I haven't check if this still works, though.
I kind of want to format your variable name in $\\LaTeX$...I think ${\\rm BLAH}$ is a much better default name than $x,\\ \\hat i$, etc....
You're predicting X from Y or Y from X? If you're predicting X from Y, the reversal of the usual direction may confuse some readers (a minor point but still worth considering and definitely worth lampshading). If you're predicting Y from X but X is measured with error, you shouldn't use ordinary linear regression for that.
Thanks a lot Procrastinator! This is exactly what I was looking for.
Hint:

$E[x(y+z)]−ExE(y+z)=Exy+Exz−ExEy−ExEz=Exy−ExEy+Exz−ExEz=Cov(x,y)+Cov(x,z)$
R has a `riverplot` package for sankey diagrams. Never used it, though, so I can't testify to its usability.
I apologize for not being perfectly clear. The point I wanted to make is simply that binary or multinomial logit models are typically not linear in their parameters. So the interpretation of the coefficients is not simply "an increase in $x$ leads to a $\\beta$ increase in $y$," as you can usually say with linear regression. If this interpretation is important for your analysis, stay away from logit models.
See [the example](http://math.stackexchange.com/questions/324370/questions-about-distribution-free-statistic/324653#comment704431_324653) I provided in your earlier question today. :-)
Is there a word missing in the note at the end? "1st column is ... and first row are names"
Could you show an example image from an other package/software or give a more detailed description what you want to plot?
... Or he may be in the middle of typing "you", which is slightly less probable, also with an edit distance of 1 (if you consider "u" to be a missing letter). Finally, he may be typing the slang word "yo" which has an edit distance of zero, but also a very low probability. p'() (as I programmed it) scores "to" the highest, with "you" in close second place.
Thank you for your time, much appreciated. If I may clarify, it is a time series. Lets say I have data for all 100 individuals over 5 years, and in the third year a dummy variable (the independent variable I am interested in) changes to 1 at the same time for all 100 individuals. I don't care about the precise effect of that change on each person, or the average effect. Rather, I only want to establish that at least 80 people were each affected. The coefficient is positive on all 100, but each is insignificant. I'm not sure measuring the distribution of the coefficient will accomplish that.
apparently this might simply be the type of pattern predicted by the logit function as in this thread: [link](http://stats.stackexchange.com/questions/70783/how-to-assess-the-fit-of-a-binomial-glmm-fitted-with-lme4-1-0?rq=1)
usually the two lines in resid vs fitted are centrally symmetric. Here this is not the case because there are not enough "y=0" responses predicted. This is my guess at this point. Any insight welcome.
@Alecos: I think the issue at the heart of the matter here is that the whole second part of the question is a non-sequitur. Integrating a function plus noise has little (if anything) to do with integrating the pdf of the noise. The OP seems to essentially be asking a question about (simple) stochastic integration.
Guido: Read up on quality control statistics. I know very little about this field, but I think it is designed to do exactly what you want (while ANOVA is not).
My initial reaction was to start with a `glm(..., family=poisson)`, b/c you have count data.  I would start with `summary(glm(Nodes_removed~(.), data=study_data, family=poisson))`, but I notice the QQ plot doesn't look so great (if you plot the glm() object you get diagnostics).  Also, there are a lot of terms in there, and I don't know what types of interactions etc. seem sensible.  With data like this, I would do careful thinking about what makes sense, b/c blindly including a lot of terms can be dangerous.
The BUGS language parameterizes its Normal distribution in terms of mean and precision - both for univariate and multivariate versions. But it would be unusual to use this notation in a paper, at least without very clearly noting that one was doing so.
I think it is a possible duplicate, or at least there was a lot of discussion before http://stats.stackexchange.com/q/25/2645 In my ranking I would suggest open source R or Gretl for time series analysis, but it's just me.
I agree with @gung . Indeed, if I had to choose between just computing an effect size (like odds ratio) and just computing a p-value, I would choose the former.

However, the odds ratio for gender will *not* tell you the odds of developing the disease in one of the genders; the odds ratio is a *ratio* of two *odds*. It measures the relative odds for males vs. females.
So to take the analogy further, we would say that estimating the center of mass is more difficult for a larger body. This makes some sense.
@HFC  Less whimsically and more formally, it is important  to not confuse between conditional distributions and unconditional distributions.  In Case II, the _conditional_ distribution of $X_2$ given $X_1$ is degenerate, and is not the same as the (unconditional) distribution of $X_1$.  But, by the law of total probability, $$P(X_2=1)=P(X_2=1\\mid X_1=1)P(X_1=1)+P(X_2=1\\mid X_1=0)P(X_1=0)=0\\times\\frac{1}{2}+1\\times\\frac{1}{2}=\\frac{1}{2}=P(X_1=1)$$ and so the _unconditional_ distribution of $X_2$ is the same as the (unconditional) distribution of $X_1$.
@RobertLong You could try http://stackoverflow.com/questions/tagged/stata
@Emilie multiply $1/3000$ by the probability of having a successful relationship. This is left as an exercise for the reader.
@Har, I have updated the answer with the definitions of $RSS_p$ and $AR(p)$, thanks for noting to that, but note that here I use theoretical model and errors with known parameters, in practice you have to estimate everything, so the conclusion on the prediction accuracy remains. Note also that $ARIMA$ models are a-theoretical usually and lacks the interpretation, so the biases  in parameters due to "wrong order" are of least concern! Primary goal is the accurate prediction.
If you're trying to determine your strongest predictors, you could interpret the plot as evidence that variables that enter the model early are the most predictive and variables that enter the model later are less important. If you want the "best model," generally this is found via cross validation. A common method for attaining this using the glmnet package was suggested to you here: http://stats.stackexchange.com/a/68350/8451 . I *strongly* recommend you read the short Lasso chapter in ESLII (3.4.2 and 3.4.3), which is free to download: http://www-stat.stanford.edu/~tibs/ElemStatLearn/
+1; as an addition, it is nice to take a look at `any()` and `all()` in this context.
1) Welcome, Andy. 2) Please check if I edited the question right. 3) Did you try do rotate (e.g. varimax) the loadings; that might help. 3) How do you hope to `select... the variables for a future diagnostic test` since you've replaced the variables by the components? 4) Why do you think you need PCA at all? Discriminant analysis itself is a tool to reduce variables, and is better and more reliable than PCA in the task to separate groups.
Can you give an explicit form of the Cholesky decomposition here? I think that the answer by cardinal achieves just that, if you think about it, by expressing $X_i$ as a function of the history.
okay now I understand your question. Thank you! Yes, by using GMR, we take the view that X and Y are jointly-distributed. Then the similar concept to "prediction interval" in LM should be the one you've mentioned... It makes sense. Any thoughts on how to obtain the 95% "prediction interval" under this new view? Thank you!
Sorry, I don't use R, but it seems as if $x[n + i - seq_len(p)]$ takes on negative as well as positive values, which it shouldn't?
Does this mean I calculate the new burn-in time using the autocorrelation and then calculate my chains again? Time is an issue for me, so I would like to assess convergence from the chains that I have now...
(+1 to Peter)  OTOH, if you feel comfortable averaging each company's 5 years' worth of income figures to create a new dependent variable, then your task becomes much simpler and might entail garden-variety linear regression.  How to conduct that in a particular program is a question for another venue, though....But have you considered whether you've accounted for all relevant variables?  For instance, maybe your analysis would be sounder if you controlled for company size at the start of the 5 years.
**Hint** Did you notice that Stata has omitted `industryM`?  *The two programs are using different codings for categorical variables.* [Searching our site](http://stats.stackexchange.com/search?tab=votes&q=%2bdummy%20%2bcoding%20%2b[regression]) yields specific advice, such as http://stats.stackexchange.com/questions/20166 and http://stats.stackexchange.com/questions/24389.
(+1) There's some code available in the Scipy [Sandbox](http://projects.scipy.org/scipy/browser/trunk/Lib/sandbox?rev=2844) which is based on Fortran code available on [Netlib](http://www.netlib.org/).
If you're asking about the line that goes down diagonally at the left side, that's when y=0, as I suggested before. It's what you often see with count data. Why not use a Poisson glm (or other suitable-for-count-data glm), though?
@ Bernd the Graham reference is extremely good, it helped me a lot with getting the hang of multiple imputation.
@Andre, I wouldn't recommend bubbles in 3D. Because the size will suggest visual perspective.
I updated my answer to show how you can do this.
Are residuals (such as Ascombe's or Pearson's) a valid measure for goodness of fit? In Generalized Linear Models (Nelder & McCullagh, 1979), they discuss them but only briefly mention them in the context of goodness of fit, without in depth discussion of such a technique.
How would I generate Yfac if my variable is already a 1/2 dichotomous variable saved as a factor in R?
+1, I can confirm that this works and it is very simple. I used it in my pretty complicated project: jQuery+PHP+R.
I found the bcp R package http://www.jstatsoft.org/v23/i03/paper which implements the Barry & Hartigan algorithm
That's a great answer. Thanks so much for your insight. Makes a lot of sense to me.
K-means has no way to create clusters, so it really depends on the initial set, but when I find a cluster becomes empty, I delete it.
A good start could be googling "linear transect", "point transect" and "capture-recapture". This book is an excellent reference: http://www.amazon.com/Introduction-Distance-Sampling-Estimating-Populations/dp/0198509278
In the last comment, do you mean $a_0=-1/2, b_0=0$? Because $a_0=0, b_0=0$ leads to a biased estimate for variance.
score test is a different name for what @jen-bohold called a Lagrange multiplier (LM) test.
@Wayne, that's my understanding, yes.  I'm not 100%, however.
@a11msp The values in "wt" are the weights passed to the glm via the 'weights' argument.
@GarrithGraham no problem, I thought this free book is a "well-known secret".
@CousinCocaine. Now we need to understand what your scientific question is. Avoid the word "significantly' which often gets i the way of clear thinking. What question do you want to ask. Clearly the different rows vary a lot, and the replicates in each row vary a bit. What calculation would be helpful?
Do you basically want to estimate the distribution of the inter-arrival time between threads?
This is the order in which the subjects are thought in most BSc Psychology courses, as far as I know. I can't see how it would do any harm, especially as "linear models" (i.e. regression) is one of the topics covered in the course. Personally, I think it makes more sense to teach regression first, before moving on to ANOVA, as a special case of regression, but there you go.
it's hard to say on a single instance because if you remove observations, even if you fix the seeds, the kmeans solution will not be the same. If you repeat the experiment 100 times (each time with a different seed), do you still get that the second clustering is worst?
Should this question really be "famous quotes about statistics"?
Thanks for the link too. I now know where the scale of measurement came from!
(+1) This is correct, but I believe the key point is that to compare a prediction to a value, it is helpful (and in many applications essential) to have an estimate of the prediction variance.  That's impossible to do with the setup described in the question.  You can get one component of that variance from the error estimates of the parameters, but there is no way to infer the covariance of the prediction residual with the rest of the data, which is the other component of the prediction variance.
Could you clarify what a "low frequency (of events?)" with continuous outcomes means?
The concept of probability can be different (in most cases they are). Also, the p-value has asymptotic properties that the Bayesian counter part doesn't necessary have. So, be carefull.
Although it makes little difference here, because the differences between the sets are blatantly obvious, it is strange to have means reported to less precision than indicated by the standard deviations. This renders the SDs almost irrelevant.
MANOVA consumes more _df_. MANOVA also assumes multivariate normality which is more strict than univariate normality of ANOVA.
I think it should be site/space since SSE is higher for that.
@MichaelChernick, thanks for your answer. However I'm still a bit confused about the way to calculate the 'mean' of the set of distributions that belong to one cluster (for the center update step of the k-means). It seems that it is possible to average the vectors of means, but what should I do with their covariance matrices?
I think it's a matter of preference.  My example is essentially what `plyr` does but it gives you finer control, though it's not nearly as clean.  My opinion would change if one solution had a better time/memory profile.  I haven't compared them though.
It is better to plot some sort of _ratio_ between the within-sum-of-squares and the between-sum-of-squares. That makes up several clustering validity criterions, such as Calinski-Harabasz, Davies-Bouldin, Gap index, CCC (mentioned by Peter). You may search this site and the web to read about them.
@cardinal I went over the possible values $Y_{ni}$ can take again, and added a floor to the log term. Otherwise, the values seem right. If I truncate at 1, I will get the values I want for $Y_{ni}$ and will be able to apply the Lindeberg condition to get convergence to the normal. However, I do not see how this will imply convergence to normal for the $\\sqrt{n} \\bar{X}_n$
Note that stepwise regression is an invalid technique unless you use a p-value cutoff larger than 0.5 or so or you use penalization.  With N=600 and 33 candidates your results will not be reliable.
@whuber: I mean whether A and C are independent, B and C are independent and so on.
@user603 I will add the material in the response, because is too large
Thanks Placidia. When asking my peers the general notion was to use the binary logistic regression menu rather than the more convenient approach via GLM. However, I still struggle to understand what happens to my data in the SPSS black box. I neither want to compare nested models nor do the number of categories increase with increased sample size. Still results make more sense using the binary logistic approach.
@ayush Could you edit your question clarifying what you mean by "certain values" and "event rate"? Could you explain what is making the situation "not very comfortable"?
Thanks so much for the very detailed reply -- I wish I could accept both answers, as they are both really helpful and thoughtful.
Do these two sets of coefficients differ?
OK, I thought this was just about understanding regression. I would look into multiple imputation.
@whuber: Have a second to chat?
Hi @Nathaniel did you happen to think it over?
@PeterFlom I think we might have some very similar experiences. I alos think you are getting some other very good responses in other people's answers.
It doesn't give them because as Greg Snow puts it: "The short answer is that you can't." More here http://stats.stackexchange.com/questions/10517/identify-probability-distributions
@whuber
Sorry I thought K means algorithm is my answer. I think distance formula is appropriate for k means. Isn't ??
What are you suggesting? I'm confused in "how to scale and re-express) thing.
See also [here](http://math.stackexchange.com/q/97250/15941) where also we find the $2^{N+1}-2^{M+1}$ result given by Daniel Johnson below.
@ Glen_b: no offense, but this is not an answer to the question. There is no $R^2$ in your output. And by the way, the `anova(fit)` will give similar out put as yours even without defining $m_0$-$m_5$! Cheers.
you might want to edit your equation to be clearer.  It's hard to see what it should actually read.  Since this seems to maybe be a homework problem too, it should be tagged self-study.
Please define the two terms by editing your question.
If the two distributions have identity covariance matrices that just means the Mahalanobis distance reduces to Euclidean distance.  So it doesn't make sense to talk about choosing one over the other.
Then I would report the regression results from the two contentments as a model apart from the two demographic variables that were significant. (in a simple regression and not together) because if I use the contentments together with the demographics, the demographic variables become non-significant
@gung: this is the same as [energy](http://cran.r-project.org/web/packages/energy/energy.pdf)
If you post links to images on imgur, chances are someone will put them up for you.
@Mike Yes, in this case (just visualizing data without numbers), you can simply express your data on a min/max scale, as is done in [parallel displays](http://en.wikipedia.org/wiki/Parallel_coordinates). [Showing numbers below the table](http://www.perceptualedge.com/example2.php) is also a good idea.
@user603 The question doesn't specify anything like that; I also don't see how it's "superficial".  It is very high level and shows how to do analysis in R.  I wouldn't recommend it as a textbook for an econometrics course, but it certainly serves a purpose.
Please note that there is an important difference between methods [clustering](http://en.wikipedia.org/wiki/Cluster_analysis) and [classification](http://en.wikipedia.org/wiki/Classification_%28machine_learning%29): for the first you don't know what your clusters are, for the second they are defined and you want to predict membership. What you mean by ““some sense”-ish point of view” may be the difference between the two.
Thanks Corey will look carefully
Could you describe exactly how the parameter "don't make a lot of sense"? How have you compared those parameters to the data?
Questions (1) and (3) are unanswerable AFAIK.  Question (2) could have some objective answers: I recommend focusing on it.
FWIW, I computed the value using this answer, using the parameters in my OP, and got 1.41. This compared to 1.44 for the U(25,30) approximation.
This is true. The choice of parameterizing the normal in terms of precision or variance (or even standard deviation) is informed partially by the algebra of the particular problem (Rubin's famous 8 Schools problem, for example, takes precision-weighted averages of the data). Either can be convenient in some context. Another consideration is programming language: in R, the normal uses standard deviation; Bugs uses precision; `stan` uses variance.
Thnaks for going to the trouble to improve the look of my answer Procrastinator.
@mbq: you missed a "good" pun there... you could have said "it's more *fan*" :)
Both use quite a lot of iterations. I considered using precomputed tables, and interpolating, but this would use too much memory with inaccurate results.
One note, I would say that predicting "out of data" (or more commonly 'out of *sample*') and "extrapolation" are not quite the same thing. *Predictive models* are always hoping to be able to accurately guess an unknown response value of a future observation for which the predictor values are known, but this is generally understood as applying to cases where the predictor values are *within* the range of values of the data on which the model was trained. OTOH, *extrapolation* refers to predicting a response value for an observation whose predictor values lie *outside* that range.
... the formal analysis broad enough to be able to treat them formally.
@Julián Urbano Tests are constructed to guarantee a specific error rate under the null. One way to check that with a simulation is to generate random data and see if the proportion of significant results is close to the nominal error level. Your data are different, you clearly have many significant results so you are looking at a strange power curve, not so much checking the test's control of the error level.
"Predicting Structured Data" talks about utilizing SVMs in the one-class(kernel density estimation) case on Joint Kernels between input and output. Given that they are both kernel machines this approach should work. I found one paper mentioning something similar. http://datamin.ubbcluj.ro/tr/2011/sogp.pdf my attempts at fitting structured learning algorithms has been pretty abysmal, so I'm not sure how this stacks up.
If you regard k as known, you could scale your populations by P/k. If k is something to be estimated, that alone means that your problem is not standard logit regression.
Thank you, I will check these out.
Thank you so much. It seems that the log transformation is widely practiced in economics. It is not part of economic theory but is widely accepted. I appreciate your very useful response.
Thanks for your reply Sirus.I mean, let's assume an investor did a transaction on date x. It was a purchase with amount of 5000 and 200 volume. I will define my vector like [5000,200,1]. (1 presents type of interaction as purchase) No, how can we organize our codes? Hope, my definition is clear.
Hillaire Belloc?  Nice work on digging that up.