tttjjj/elt_bench huggingface dataset:
$(X'X)^{-1} X'Y$ is the least squares solution for the coefficient vector and is unrelated to your question. The SEs for logistic regression coefficients are not expressible in closed form but you can get approximate standard errors using the fisher information matrix. I think these standard errors are what's outputted with standard statistical software. You can calculate the AUC using its close relationship to the Mann-Whitney U - see [the wikipedia page](http://en.wikipedia.org/wiki/Mann%E2%80%93Whitney_U#Area-under-curve_.28AUC.29_statistic_for_ROC_curves) for more info.
In addition, the authors of this work are happy to share the R code of their implementation with you. Just email them. It's what I did.
@Glen_b: That's indeed true. Starting from scratch, I would also try to go for an appropriate GLM with S-shaped link function. But given the review status of the manuscript, a quick and dirty solution also has its advantages.
Not sure that an epoch can be found to suit every outcome but for any practical problem With a non-stationary signal you need to find someway to take into account temporal variations (if a signal/time-series adequately described analytically or is linear time invariant this not necessary). The epoch length is again domain specific but usually chosen to be short enough that the signal is stationary under the time period (epoch) of interest.
I think that's figure 2.4 from [The Elements of Statistical Learning](http://www-stat.stanford.edu/~tibs/ElemStatLearn/) where they compare nearest neighbors with regession-type methods (and of course they provide multiple comparison points throughout the book, too).
@Legend: Yes, the JS divergence is symmetric. Hopefully that is easy to see from the equation given in the answer. Make sure that you are taking the KL divergence between $P$ and the midpoint measure and $Q$ and the midpoint measure for each of the two terms. Separately, neither will be symmetric, necessarily.
The `lavaan` package works with qplot to create reasonable diagrams.
Thank you very much for your answer, rolando2. Yes, like you mentioned, meeting all my criteria may be difficult and I may have to leave out some cases which I don't want to do. I'm meeting my supervisor next week and let's see if she can save me! Thank you all very much for your help. You have been amazing!
@NeilG - thanks! I just noticed that the OP switched from "from a complete population of $m$ elements are drawn ... only the $n$ smallest are known" in the text to $m < n$ at the end.  I'll clarify which notation I'm using in an edit...
Several related questions (none with satisfactory answers): (**1**) [How to sample uniformly from an intersection of simplices?](http://stats.stackexchange.com/questions/12645), (**2**) [Random matrices with constraints on row and column length](http://stats.stackexchange.com/questions/17633), (**3**) [Generating random matrices with specific equality constraints](http://stats.stackexchange.com/questions/19173), (**4**) [How to generate a bounded random correlation matrix?](http://stats.stackexchange.com/questions/10450)
I would say that cross-validation is a good approach, since it is generally applicable and prevents overfitting---as long as you perform every parameter selection __within__ the cross-validation loop. As for grid search: see my comment to @Sentry's answer below. More in general, you could try one of the many [Meta-Heuristics algorithms](http://en.wikipedia.org/wiki/Metaheuristic), such as [evolutionary strategies](http://en.wikipedia.org/wiki/CMA-ES). These usually work quite well---especially if you have many different parameters to set---but they are not always trivial to implement yourself.
@ chl where would these questions go though? I found it quite hard to find references on setting up R, sweave and latex when i started, so thats why i answered.
I recommend that if you are serious about econometrics modeling that you get over your phobia and learn R, SAS, SPSS, etc. To complain about programming is the equivalent of an accountant complaining of having to learn arithmetic. If you're doing your analysis with Excel, you're lacking in professionalism. There have been enough media stories about economists and pundits embarrassing themselves by (incorrectly) using Excel to do their analysis.
No, that doesn't resonate.  Box-Cox is used for continuous univariate $Y$, and many users of Box-Cox do not know know to penalize for uncertainty in $\\lamba$ nor that Box-Cox makes a strong assumption about the measurement origin (zero).  Splines do not overfit any more than having too many predictors, and you can control the amount of fitting with the number of knots and with shrinkage (penalization; see the R `rms` `lrm` function for quadratic penalization).  As you said, it is best to estimate transformations in an adjusted rather than a univariate fashion.
Some time series can be stationary to begin with. The I in ARIMA stands for integrated which actually means taking differences. This is done when there are polynomial trends to remove to get stationarity. All this is cover in detail in the [Box and Jenkins text](http://www.amazon.com/Time-Analysis-Forecasting-Probability-Statistics/dp/0470272848/ref=sr_1_1?s=books&ie=UTF8&qid=1346697271&sr=1-1&keywords=box+and+jenkins) as well as other time series texts.
@Anne Using 2, 7, etc., is likely to be biased within most age groups, depending on the pattern of the age distribution in your population.  This would result in a net bias of around -1/2 year in many calculations.  In lots of applications that's an unimportant error; in other applications it might matter.  It wouldn't change the estimated standard deviation at all.  The biggest problem with using 2, 7, etc., is it indicates a lack of understanding of what the data mean, which would raise red flags in the mind of a critical reviewer of the work.
I wrote a grant proposal once to work on software for this, but it got turned down.
You are correct, but might be unaware that your answer could easily be misread as invoking an incorrect principle: namely, that the chances of individual outcomes must be equal. This "principle of indifference" is the source of many mistakes over a very long period of time. What is needed to shore up your argument, even though it's a trivial observation, is that because there are equal numbers of even and odd balls in the box, *therefore* the eight outcomes each have equal probabilities.  (The axiomatic justification for this implication follows from the independence of the draws.)
@cgillespie: Concentrations, yes; but age?  That is strange.
+1, thanks for the links.
(+1) Not sure about answering questions specifically but I believe the time it takes to say the same thing is broadly the same in many (all?) languages. Wordy languages tend to be spoken faster and vice versa.
Hi @January, I was concerned that the "importance" or the relationship between the principal components and the variables might change when huge number of variables are included. I have applied PCA before for a relatively small number (100) variables, but wasn't sure if I can still trust the method with this large number of variables. Thanks for your reply :) Can you rewrite it as an answer please?
Wouldn't H0 be: there is no link between passive smoking and CHD? Since the null hypothesis is usually the hypothesis that there is no effect. Apart from that though, thanks for this answer!
Don't bother with neural networks, this is an obsolete technology.
@RichardB: Did you read this: http://stats.stackexchange.com/a/34725/4598 ? I'm still not clear what your "axis" is. If it's the loadings or rotation matrix, then it contains the eigenvectors. If you are not sure what your black box software spits out, you can a) calculate the same problem with another software where you know what happens, and/or b) have the black box do a PCA on a small data set where you can calculate the PCA by hand. This should allow you to find out what is going on.
I do not agree but I do like the metaphore ;)
Nevermind, I just got it. For $df = 1$, the Chi-squared distribution is the same as a single squared normal distributed variable, hence the equality. I really should read up more on statistics ;)
The edit is still inconsistent. The table shows no entry reflecting "answer = 3."
I'm confused about what you say in #5 about fixing the covariance to 1. Surely you meant correlation and not covariance (unless both variables have a variance of 1), correct? Also, if you did mean setting the correlation to 1, it seems that the two variables would have effectively been reduced to a single variable (and not merely been put on the same scale), given they would always take the same value
The last page comes from this article: http://www.math.wm.edu/~leemis/2008amstat.pdf.  I've posted a condensed version with links for footnotes here: http://www.johndcook.com/distribution_chart.html
I quite like the Bayesian approach, I probably can generate a probability distribution. I need to do something SEM maybe the correct technique but the number is patently meaningless. The original answer, well answers what I asked, which, after a quick google seems to be a common issue.
If this answer covers your question completely you could mark this answer as accepted.
@Denis The intention of my thesis is to show that my codes are faster than LAPACK.
That's true - thanks. We can assume that $cd\\geq 2$.
A good aside question here is 'why'.  Why use the BLUP as an indicator of how expensive the district is rather than conduct that analysis separately (or just with descriptive statistics).  However, as a general point of knowledge it is a good question all the same.
Thank you.  Knowing the name of the problem is quite helpful!
Can you provide some additional information or documation that supports your answer?
@Henry For background you could read the *almost identical* series of questions [here](http://stats.stackexchange.com/q/11315/919), [here](http://stats.stackexchange.com/q/11351/919), and [here](http://stats.stackexchange.com/q/12262/919).  ["What we've got here is failure to communicate."](http://www.youtube.com/watch?v=SnO9Jyz82Ps)
Equicontinuity is only interesting for a family of functions. If you're only looking at one function, then it's just plain old continuity.

Are you asking if a line is continuous? Yes, it is.
Your question will not be understandable to anyone without reading through your code, so if you would like a good answer then please edit it to limit the code to a small section that seems to be causing problems and please explain what it is intended to do.  Are you aware a simulation is not only unnecessary but inefficient?  [Exact answers](http://stats.stackexchange.com/questions/24385) are easy to compute.  More solutions are also available at http://stats.stackexchange.com/questions/3614/how-to-easily-determine-the-results-distribution-for-multiple-dice.
Dave, mathematics is a more reliable teacher than software. In this case `R` computes the unbiased estimate of the standard deviation rather than the standard deviation of the set of numbers. For instance, `sd(c(-1,1))` returns `1.414214` rather than `1`. Your example needs to use `sqrt(9/10)*sd(x)` in place of `sd(x)`. Interpreting "$\\sigma$" as the SD of the *data* and "$\\mu$" as the mean of the *data*, your BTW remark is wrong. A program demonstrating this is `n <- 10;
x <- rnorm(n,5,2);
m <- mean(x);
s <- sd(x) * sqrt((n-1)/n);
m2 <- sum(x^2);
c(lhs=n * (m^2 + s^2), rhs=m2)`
If your goal is to find out which region your brand is doing better (selling more) then why are you comparing last month to last year? You should be comparing across brands.
Thanks.  I'll look up LASSO. (It looks like it's mentioned in the reference cardinal gives, too.)
Thanks JulianHi for your response. Actually, I'm using the caret package for a classification problem. These values are indicator variables/descriptors. My output is a factor of two levels.

Cheers
classifying everything as belonging to the negative class isn't a wierd result, sometimes that is the right thing to do because the density of negative patterns always exceeds the density of positive patterns everywhere.
Although in some cases it may be meaningful to average a set of correlations, as suggested at http://stats.stackexchange.com/questions/8019, in other cases the results would be nonsensical (as I argue in a comment at http://stats.stackexchange.com/questions/8019/averaging-correlation-values/8023#comment13152_8023). Given that you have the original data, there are better ways to proceed (as suggested in many of the related threads).
There's been discussion of adding reversible jump MCMC to OpenBUGS per page 288 of *The BUGS Book* and WinBUGS has, I believe, a Jump module. Anyone heard of any similar development for JAGS? Is there some way of writing JAGS code to get the effect of doing the varying dimensionality? Is it simply a matter of defining dimensions larger than needed?
@Android3D Because you already posted this answer at http://stats.stackexchange.com/questions/73320/how-to-visualize-a-fitted-multiple-regression-model/82396#82396, I am converting it to a comment here.
@dsimcha But the normal distribution is just a probability density function which could predict the number of observations in a given bin of the discrete variable. So, I would understand if you said "no real variable is exactly normally distributed and this is why normality tests will fail at some point". But for "discrete data cannot be normally distributed since it's not continuous" I'd like some reference. I'm really interested in that kind of stuff. Not wanting to start a fight here.
Please edit this post to change "model fitting" to "model selection". It would also be helpful to discuss which method you're using. For instance, if stepwise model selection based on p-values is used, then stacking imputed data is absolutely NOT allowed.

You can draw bootstrap resamples of your data, including missing data apply MI and the subsequent model selection process and calculate an exact "p-value" for the selected model.
I don't think this is duplicative (of that question, at least). @Sandi's question asks about interpreting negative structural pathways in the presence of negative factor loadings, which is a special case of this more basic question (i.e., how to estimate structural pathways in the first place). Momo's other suggestion, Judite's question, is also about a special case of regressing a manifest variable onto a latent variable, which is sort of halfway between SEM and path analysis (in as much as these are distinct), but again, the question is not simply about how to do it, as haroldcampbell's is.
You should not cross-post without linking. This wastes the time of the people trying to help you. http://metaoptimize.com/qa/questions/12024/is-classifying-documents-according-to-their-topic-useful-for-any-application
(+1) This is a helpful way to describe an abstract and difficult concept. I believe, though, that the phrase "...is no bigger..." should be "is no smaller." Better yet, that section could be made clearer by removing the negatives and using a parallel construction, as in "The sigma algebra generated by two variables is at least as large as that generated by one random variable ... So the information about $Y$ contained in $\\sigma(X,Z)$ is at least as great as the corresponding information in $\\sigma(X)$."
@VitalStatistix, I think your answer is just a rephrasing of my question. Do you have a reference or proof to back up your answer?
I recommend looking at the standard environments at http://library.rl-community.org
Is this what you mean?  http://www.math.ucla.edu/~tom/Stopping/sr7.pdf
I can be more modeling specific, but I don't want to guess here. If you chose to modify your question, I will add more details.
For what purpose are you testing?  (There is an *enormous* amount of literature on this, by the way; my question concerns whether any of it is really relevant to *your* problem.)
As you ask, quoting a chi-square statistic to 14 d.p. would at best be considered a rather strange joke and at worst as strong evidence of not thinking! I wouldn't give more digits than 20.578. That's still more than anyone needs, but more conventional. The number of degrees of freedom for this test is 3, as you said earlier, as (#rows - 1)(#cols - 1). Note that chi-square tests appear in all kinds of statistical problems; this is just one of the two kinds met first in many elementary courses.
Because this is a homework question, you should add the `self-study` tag. See http://stats.stackexchange.com/tags/self-study/info
If by *data mining* you mean something closer to *machine learning*, then you might look at Sebastian Thrun (used to be at CMU; now at Stanford and Google, I think) and collaborators designing self-driving cars (I think they're now at least partly funded by Google, but it started with some DARPA challenges) and Andrew Ng doing other robotics tasks incorporating computer vision. He also has a nice example of inferring depth (i.e., third dimensional) from 2D still images. There is also a tremendous amount of research by many people in scene recognition from still images.
@Dilip: Not only is there no known exact formula, better yet, it is *known* that no such formula can exist!
Please note that the setting is that of a *multivariate normal distribution*: otherwise, your assertions about (in)dependence are not true.
Can you reformulate your desire for a hypothesis test as a *question to ask of your data*?  We might then see what would be most appropriate.
How about the variance implication when approximating binomial by Poisson?
Currently in this shape the question cannot be answered. Can you explain a bit more? From your question it seems the dependent variable is test results and "performance", while the independent variable is duration. First, how are the test results presented? Are they continuous (like blood pressure, glucose) or binary (like hypertension y/n, or diabetes y/n?) Second, same goes for performance, how is it measured? and third, why did you mention age, gender, and questionnaire? How do you plan to use them?
An important phrase here *"the standard deviation of the mean"* which is *not* the same as the standard deviation of the distribution.
Hi Drew, sorry for my late answer. Well, in case of zero frequencies, it is advised (at least I was taught so) to either merge categories, since Chi-square allows some 0 frequencies (less than 20% frequencies can be 0 when the factor has more than 2 levels), if that's reasonable, or to drop Chi-square and choose an alternative technique. As I usually drop the thing, so you may want to ask another question regarding appropriate goodness-of-fit test.
"I cannot guarantee..." sentence is hard to understand. Give more details that would be in line with SPSS.
It would be much easier to interpret the book's meaning if you provided a larger quote with some of the surrounding context.
@whuber: Thank you for the critical appraisal. I'm afraid, when it comes to mathematically _proving_ something, I sink. I've reported part of my _practical_ experience (I said that); the answer wasn't really an analytical sequence. Wouldn't you like then to add your own answer that can correct/improve mine? It might turn out a valuable aid. Or, you are free to work on my text to improve it if you find it not downright futile.
+1 But I think it's more accurate to call BUGS a **declarative** language, not a functional one.
And regarding the difference between factor and fixed factor, I would use these as synonyms here. But there's a chance I'm wrong about the terminology here.
Thanks for your answer. This question is part of a problem set for which they have told us to specifically use OLS, but I'll take note of your point for my future work! Thanks again.
"Support" is the set of values that the random variable can attain; that is, the set of values where the probability distribution function (or the probability function for a discrete distribution) is nonzero.
-1 This link does not appear to offer any insight into the question. Is this spam?
@juliohm. Yeah the random field is assume to be $T(x_0)$ + random noise $\\epsilon_i$.  I wanted to know how the weights will look like in this case when I want to estimate the underlying value $T(x_0)$ insteald of $Z(x_0)$
Note that the caption to the original figure is "Univariable lowess smoothed logit versus AGE."  This indicates it is a lowess smooth of the binary response data re-expressed on a logit scale.  The purpose, as stated in the text, is to evaluate whether a model that is linear in the logit of AGE makes sense.  Based on the linearity from ages 40-56, and the very rough linearity for all ages, the conclusion is that linearity is reasonable.
I am not quick in deciphering man-machine language. Does this mean that you regressed, say $V_2 = a + bV_5$ and then calculated the correlation coefficient $Corr (V_1, \\hat U_{2,5})$, where $\\hat U_{2,5}$ are the residuals from said regression??
That's was a very intresting read.
"sketch" means "draw a picture" (usually by hand)
I find it unfortunate that this question received two downvotes (as of writing this comment). While I do not necessarily agree with the respondent in this context, it definitely deserves some comments as to why it is inappropriate enough for a downvote.
Do you have any references to pros and cons of linar and tree GP?
I don't believe this is correct (But please correct me if I'm wrong :) . See my edited question.
It comes from the Poisson approximation and solving $.99 := P(X>0) = 1 - e^{-np}$
@Peter, you might have a difficult time even *defining* randomness if you could refer only to the generation mechanism. Ultimately, because all the utility of random sequences lies in the numbers they contain--and not in how those numbers were produced--there must be a way to define and test randomness purely in terms of the sequences, don't you think?
This sweeps some stuff under the rug concerning when you hit the next $x_i$ value, but is close enough to the handwavey proof desired, I think, at least for the median.
thanks for the idea. I'll try it, but I expect variance to naturally decrease with age in this case.
PCA does dimension reduction within a set of variables. To do dimension reduction that maximizes correlation *between* sets of variables, you may want to look at *[canonical correlation analysis](http://en.wikipedia.org/wiki/Canonical_correlation)*. (In the case where you have a univariate $y$, finding the linear combination of $x$ that best predicts $y$ would basically just be regression, wouldn't it?)
Sir, the answer is simpler when we talk about the binomial experiment. The above mentioned texts as we move on never provide the motivation for the distribution. I intend to ask this question on many distributions, individually in fact.
Ok, I had to leave for lunch and thought the type 3 sum of squares was THE answer. However, I just had a chance to do the ezANOVA, as suggested. It ran, as specified, but I still get the same F and p effects as I do get with the aov function and the within subjects effect differs from the F and p that SPSS gives me.
@madCode I've never actually used vowpal wabbit, so I can't help you install it.  I've heard their mailing list is excellent, and I'm sure you can find help there for setting it up.
Thank you so much for clarifying the notation!
+1, clear, well thought-out question. If you just want a bar chart, you can solve for the predicted mean response for each of the 3 treatment conditions at the mean of the covariate.
parallel coordinates sounds like another one to mention.  Also methods of dimensionality reduction might be helpful.
I didn't say $E$ doesn't have a variance, only that I cannot *define* a variance for it by $S^2$. So the question is whether I can define an error *at all* and if $\\bar{S}^2$ is a reasonable candidate. By statistically independent I mean that the $E_i$ are obtained using different random numbers, e.g. by using differently seeded random number generators (I hope thats the right term then).
Correct, after you sample from the copula, you transform the margins individually, in the way you suggest.