tttjjj/elt_bench huggingface dataset:
@FrankHarrell a little simulation can prove that the correlation between the p-values (presuming you're talking about the F-test or equivalent) and the AIC is nonexistent (0.01 in my simulation). So no, there's no relation between the P-values and the AIC. Same for BIC and Cp. Another little simulation will also prove that one gets pretty different results in a stepwise procedure depending on the criterium you use. So no: Cp, AIC, BIC are in no way just transformations of P-values. In fact, if looking at the formulas I can in no way point to a mathematical link or transformation.
Sometimes the outlier are where you should drill for oil.
+1 This is a really good answer.  I'm sorry it hasn't been noticed before.
I can't see any reason this would *not* make sense. The average earnings (or whatever) of a region is a fact about the city, just like the earnings of a person is a fact about the person.

You can compare one to another.
It might be the correlation between random intercepts `(Intr)` and `Group`, i.e. larger intercepts are correlated with 'smaller' values of `Group`.
You can have a look at [Multivariate multiple regression in R](http://stats.stackexchange.com/a/11132/1909) for a walkthrough including multivariate tests.
thank you for your answer!
If you have the proportion of the trait and the sample size you can calculate that. See [this](http://onlinestatbook.com/2/estimation/proportion_ci.html) for details. Once you have figured out, you are welcome to answer your own question.
Unless you can say more about the process that determines the winner in a battle, this question has no right answer.  In fact, there might not exist a ranking at all.  For instance, the players could be rolling [intransitive dice](http://en.wikipedia.org/wiki/Nontransitive_dice) to determine the battle outcomes.
@IrishStat - yes, those are all good points and I wasn't trying to address the full set of issues, just draw attention to the problems of using the trend component from the output of R's stl() as the response variable in a regression.  stl() uses locally weighted regression in its decomposition which generally gives sensible results when it comes to trends changing direction, etc., although of course it has limitations compared to model-based methods particularly for forecasting.
No. You can (and should) look up the differences.  This will be easier if you spell them the way I did.
This is not a simple problem because if you start with a 2, your next 2 rolls can be 2, 2, and then you're looking at the next *4* rolls, and so on. That, I think makes it not just a simple geometric series problem, but a variant of a [branching process](https://en.wikipedia.org/wiki/Branching_process), and I think it's probably best solved with [generating functions](https://en.wikipedia.org/wiki/Probability-generating_function). See, for example, sec 3.7 [here](http://www.am.qub.ac.uk/users/g.gribakin/sor/Chap3.pdf), but you can find a lot more on this approach.
So why is +1 needed in R and not in the usual tables?
As I understand, value does not matter. However, for example If a test document has a lot of unknown terms, high value for unknown terms affect results as reward; low value for unknown terms affect results as penalty. Doesn't it? Can proof it by using equations?
Do you have a single website that you are changing/manipulating yourself or are you testing a collection of real websites? Will each customer/participant see all website variants or only one?
@whuber Or maybe a bootstrap test?  I think another point to make about the F test is that it often requires large differences for the difference to be detected (for commonly used sample sizes).
Thank you Mr. Flom. You've helped me with a good number of options to try.
The model will be nonlinear in $A$, $B$, etc, but the regression will still be linear in the variates $A$, $A^2$, etc.  What you will want to do in R is to define a new variable "A2 = A^2", then use the command "lm(formula = T ~ A + S + V + A2)" (for example).
I again thank you, but here is a quote from "Multimodel Inference: Understanding AIC and BIC in Model Selection", by K.P.Burnham and D.R.Anderson,Sociological Methods Research 2004; 33; 261 

"Akaike found that the maximized log-likelihood value was a biased estimate of EyEx[log(g(x| ˆ θ(y)))], 
but this bias was approximately equal to K, the number of estimable parameters in the approximating model, g (for details, see Burnham
and Anderson 2002, chap. 7). This is an asymptotic result of fundamental importance."
"The predictor variables are not linear." doesn't make sense in isolation - in order to be "linear" they must be thought of as a linear function of some other variable which you said nothing about in the question. In any case whether the predictor is "linear" has nothing to do with whether the model relating it with a dependent variable should be linear or not.
Please explain why you cannot solve the homework and which level of help you require.
Transform your data to the whole real line, do the forecasts, and back-transform.
Also, are these paired data, that is the pre-post values are measured for the same subject? If yes, you need some idea about the within-subject correlation.
On your first suggestion: My data is kind of noisy; so I'm reluctant to lose some signal during the initial correlation elimination or during PCA. If I'd do some variable reduction, I'd like to do it based on the signal.

In this particular data set I'm working on, the variables aren't void - but I'll try your second suggestion on another data set - didn't know about this algorithm.
With 3 million observations, and 60 features, there is no ostensible reason to perform variable selection, unless you are running into numerical problems. Particularly if you are interested in prediction.
The "problem" with the rug plot is just that one tick and several ticks in the same place look the same. You could use a spike for each distinct value whose length was proportional to number of values, i.e. a histogram with bins the distinct values.
+1, nice answer!  I like the clear theoretical exposition.
Thanks for the detailed explanation. It definitely clears up the mystery. I asked roughly the same question on SO (the link that you provided), but the answer I received there was focused mostly on the shrinking heuristics.
@yue86231, when you have an interaction term in the model, the main effects (ie, research & gender here), are the slopes when the other variable is 0. It might help you to read my answer here: [What does "all else equal" mean in multiple regression?](http://stats.stackexchange.com/a/84316/7290)
(1) From the model alone, it isn't obvious that the coefficient ordering is desirable, but *substantively* it is reasonable. In a repeated measures clinical trial, for example, there is no substantive reason to expect that a small perturbation of $Y_{j-2}$ to stochastically influence $Y_j$ more than a small perturbation of $Y_{j-1}$. The ordered LASSO makes better use of this a priori knowledge, with the minor risk that it might be not be true.
Since this is time cross-correlations are impacted by auto-correlative structure within the two series . See http://stats.stackexchange.com/questions/27748/relationship-between-two-time-series-arima/27756#27756
@gung Sorry, I missed the homework tag.  I answered because it didn't look like homework. When I suggested ANOVA I had forgotten for the moment that the response was binary.  I guess this should be handled as a 2x2x3 contingency table.  i will edit my answer.
I think you are making a spelling error: `thresh` not `tresh`
Pages 196-197 of this [document](http://www.med.mcgill.ca/epidemiology/courses/EPIB654/Summer2010/Uncertainty/Koerkamp_2010_Patient%20heterogeneity.pdf) might be of interest. These seem to be part of the medical statistics jargon.
In respect of the question being asked, "what can be said about the importance of optimizing the "correct" loss function?" my example addresses that question very clearly and directly by showing that the two can be arbitrarily different.
@lindelof Yes, you're right. In the first $2_{\\text{III}}^{5-2}$ 8-run design, all main effects will be aliased with a least one first order interaction. I'll try to elaborate more in a response if I find time.
my goal is to understand if that formula is good or not
Although there is a proposal on Area 51 for a Stack Exchange site that would cover questions of this nature: http://area51.stackexchange.com/proposals/37195/data?referrer=MM0IewN_HrmFRGSmvGLajg2
I will definitely report the values and tables but I would have liked to insert a diagram before the regression results aswell as a visual aid to explain what I am proposing.
It is confusing because including a new value changes the mean, so all the residuals change. It is conceivable that even when the new value is far from the old mean, its contribution to the SD could be compensated by reducing the sum of squares of the residuals of the other values. This is one of the many reasons why rigorous proofs are useful: they provide not only security in one's knowledge, but insight (and even new information) as well.  For instance, the proof will show that you have to add a new value that is *strictly* further than one SD from the mean in order to increase the SD.
The key issue here is to decide how you want to *weight* the various scores (A, B, & C) relative to each other. Eg, you might think that someone having a better rank on A is 2X as important as having a better rank on B; alternatively, you might think they are equally important. Can you provide that information?
Does the solution need to be in R?
I rolled back @Drake's last edit. Drake left a negative message, which you can find, but at least in my view Drake's answer raised points of interest, which I suggest should remain.
@EngrStudent: the comment was not meant all too seriously, see my answer below. It shows that a full singular value decomposition possibly might be feasible due to the relatively small second dimension. If not, you can try an online SVD (maybe with random data access) and hope for early convergence ... I guess it will converge early if the points you mentioned are satisfied.
@chl, SE still has an issue w/ searching under old tags. (See my meta-SO Q, eg: [how can you find questions with SE's search features that have now defunct tags](http://meta.stackexchange.com/questions/135466/)). Since 'books' was made a synonym for 'references', when you click on it you can only search 'references'.
I have made some substantive edits to your question, because I think you were getting your IVs and DVs confused. Please review to check that these changes are correct.
I believe this argument, although it's on the right track, is invalid: you *must* consider the double limit $\\lim_{H\\to\\infty, J\\to-\\infty}\\int_{J}^{H}xf(x)dx$. It is possible for the single limit you consider to exist for *all* $k\\gt 0$ yet for the double limit not to exist.
This question appears to be off-topic because it is about how to save SPSS datasets.
Could you give references? The only concept I can think of is the error variance, which clearly does not go away when N increases.
Just to make sure, is $\\exp \\big \\{\\frac{\\theta^2}{\\sigma}\\big\\}$ correct?
(+1) The "process" point of view seems to be a bit of a red herring here. Write $$P(x) = \\beta_1 \\sin x + \\beta_2 \\cos x + \\frac{1}{2} (\\sin x + \\cos x) \\>,$$ where $\\beta_i = \\alpha_i - 1/2 \\sim \\mathcal U(-1/2,1/2)$. Then, for each fixed $x$, the first two terms determine a *trapezoidal* density function and the last term is just a mean offset. For the determination of the trapezoidal density, we need only consider $x \\in [0,\\pi/2)$.
Thank you for pointing that out! That was a mistake from my side, using the wrong array as an input.

I edited my post above now and the error still prevails even after using the right input.
@whuber : I am doing a GLM where rainfall is my independent variable and plant growth my dependend variable. When plotting them against each other it seems that an unimodal distribution is present. I got then told that I have to square the independent variable to create a fitted line but I dont know why..
Your correlation matrix is ill-conditioned (the [condition number](http://en.wikipedia.org/wiki/Condition_number) is 335.9, an order of magnitude larger than [values usually considered to indicate multicollinearity problems](http://www.nd.edu/~rwilliam/stats2/l11.pdf)).  Because this contradicts one of your important assumptions, you might want to re-think your question.
Unfortunately, this is a simpler answer only because it is describing a different type of "nested model" than the type the OP is asking about. The OP asks instead about nested models in the context of *hierarchical / multilevel models*. That is, this answer, while correct in its own terms, is incorrect in the context of this thread.
By infinite data, OP means the distributional assumptions underlying the model. Eg with linear regression, we assume that the response has a normal distribution with mean given by a linear combination of the predictors. This lets us prove various results regarding unbiasedness, consistency, etc as the sample size increases without limit. I'd be interested as well if there's any similar theory regarding SVMs.
@Glen_b, how can I do that calculation of the derivative in R, I´m really new to this all, and most of the job I´m doing in R is using Rattle.
Hi Peter, 

Thx for the reply. Actually I did try a k-means clustering beforehand and found out that there are some clusters. But I was looking for a more descriptive definition for cluster centers and thought EFA might help. So I applied EFA for the total data set.

Do you mean I should do the EFA for each cluster separately?

I am not familiar with the plots you mentioned I will do some reading on that. Thx.
@Xi'an: No. For an array of coefficients of length 2, len(a)=2. Therefore, the line "for j in range(len(a))" iterates j=0 and j=1. On the other hand, the iteration is done only if i>=2, so in the first iteration i=2. Therefore, s=s+a[j]*AR[i-j-1] for j=0 is s=s+a[0]*AR[1], and for j=1 is s=s+a[1]*AR[0].
Great effort whuber and very useful information,thanks. Actually, the effects of weekends,vacations,people activities have been put in consideration in other analysis. My real data were measured hourly during the year meaning that I have 24*366=8784 points. In the above figures, P and T are the average point for each day.For each day, average point for P and T were calculated and plotted.T is the weather temperature only, it is not the cooling or heating temperature.Also,can we forget about the characteristics of P and T and deal with them as points Y and X and study their figures only, thanks
@Jase: Right now, no! I would suggest you to read p.313 Applied Econometrics Time Series (Paul Enders, First edition).  Enders suggests to start with 12 lags for quarterly (unlike 4, in the above answer). His argument is based on the theory and data availability. For example, if there is theoretical justification that the variable may have influence up to two years (and provided that there is data for, say like 30 years) one can start with maximum lag of eight). Where there is no clear theory, one can use max lag length of 4 for quarterly data.
For graphics you might find the R functions **splineplot** and **cdplot** helpful.  Indeed if you just wanted the fitted lines and didn't need a lot of regression machinery then you could probably coax **cdplot** into giving you the relevant curves (it's only **density** underneath)
+1 and thanks a lot for your answer, but why do you use Cov(Z,Y)/(Cov(Z,X)) and not Cov(Z,Y)/(Var(X)) ? Where did you get this first equation, that beta = Cov(Z,Y)/(Cov(Z,X)) ?
References please. More importantly: 1. There can't be a "correct number" here independently of the kind of distribution you are drawing from. 2. These rules usually come from interest in short-cut methods of estimating the SD from the range. Now we have computers.... Do you want to do that and why? Why not just use the data?
You've got a lot of questions in 1 here.  I've answered the first part about the 2 but your later question would require a whole additional answer.  Perhaps divide these up because the repeated measures v. biased estimators are two separate things. (not everyone uses the biased estimator for Cohen's d... Cohen never defined it).
Yes, but on the same page you find a link to a possible solution ;-)
@Peter this is strictly theoretical. I'm following a class right now and the prof uses MLR this way and I'm thinking that might be stretching a little the notion of a single outcome having "mutually exclusive categories". You're right for the 4th category, I'll do an edit for that.
Possibly relevant: http://www.msi.org/publications/publication.cfm?pub=1974 "authors Rooderkerk and Pauwels investigate what features and characteristics determine the number of comments that a post receives on an online discussion forum. The study is the first in-depth statistical analysis of the behavior of members of the focal discussion group."
What about Fixed-effects Vector Decomposition?  There seems to be a lot of controversy on this method, however.
Thanks for your reply. And sorry.
(similar to Erik P, but a different brand) We do this all the time via LiveMeeting -- you share your desktop, but can switch so you're looking at somebody else's desktop.
For reference, this Q was asked on SO where I attempted an answer: http://stackoverflow.com/questions/5648383/clustering-in-r It would appear the OP has not attempted to improve the quality of their question in the meantime
So, this is just a restatement of the standard result when the model is misspecified.  This result has been derived and re-derived many times.  The clearest and most illuminating derivation I have seen is from Kent 1982 "[Robust Properties of Likelihood Ratio Tests](http://www.jstor.org/stable/2335849)" (Biometrika 69:19).  However, you did not answer my question.  My question was specifically about Wilks 1938 proof, and why it fails.
It is impossible to tell from the information given. Also, Poisson regression does not assume that the mean and variance are equal, it assumes that the *conditional* mean and variance are equal.
+1, very good answer. Better than mine, since I answered what OP asked, and this is probably the answer to the question OP wanted to ask.
I wasn't totally sure about this, but this thread comes to similar conclusions:  http://stats.stackexchange.com/questions/13597/on-the-strength-of-weak-learners
@jefflovejapan, we can back out function of $x$ from the conditional expectation, because sigma algebra generated by random variable $f(x)$ is a subset of sigma algebra of $x$.
Thanks for this useful reference. In their article, Edwards and Allenby unfortunately collapse when a product choice is not offered and when a product is refused.
sorry I meant covariates
This is a good start to an answer to the OP's question, I think. The notation should be clarified somewhat, particularly the meaning of $\\Gamma[t;\\,1,v]$, and the OP's specific queries regarding symmetry (for what values of the parameters?) and the finiteness of $\\mathbb{E} e^Y$ should be addressed. There are several freely available references on using these models to price options, so providing those in the answer as well would go along way to wrapping up all loose ends.
+1 for a "native" R solution
Why are you talking about 'getting new data' there? I'm talking about avoiding a problem with the available data by dividing the existing data into those two parts differently. Do you mean the testing data *has* to be the last $\\lceil n/3\\rceil$ rows?
@Wiener.Snausage I guess it would be more appropriate to say that there is very little information in that case.  It is possible for Max=n and Min=1 for any n>=k>=2.
Can you share your purpose in comparing the fit of the different models? To develop a predictive model? To describe a particular data set? To provide evidence for or against a particular theory? You might choose different comparison approaches depending on your purpose.
We might be able to rescue these data instead of consigning them to the statistical junk pile. Is there substantial overlap among the items in the two ratings? If so, perhaps those could be used to calibrate the two tests. (Or, more simply, you could confine your comparisons to common items in the two ratings.) Calibration would require making (and testing) some assumptions and introduce some uncertainty, but it might enable you to draw some kind of valid conclusions. The chief difficulty is dealing with the confounding: all differences could be attributed to the change in scenario.
I must say revisiting this topic many years after having studied it using a non mathematical introduction has been very confusing at best.
Interesting that we cannot assume that you know anything much, but you assume we know what MMA means. Googling suggests "mixed martial arts".
@FranckDernoncourt, it's unclear to me if the symmetric matrices you are simulating are necessarily positive definite.  Do they always have positive eigenvalues?
...but what if I compare models with just 2 vs models with 5 or up to 10 parameters? Ideally we'd be after the simplest model - however, with large -2*LL values the penalty (2k) for more compex models is relatively small and hence those more complex models seem to rank higher than simple models?!
what type of data are you working with? From your examples I assume you are dealing with time series?
@Ryan How exactly does the t distribution apply here?
Please use proper sentences and explain the problem in more detail. I've no idea what you are trying to do based on this description.
And here I thought I had proofread my text thoroughly.. I have changed this to "for various properties". In any case, the values of the properties can be pretty much anything, such as integers, ranges, strings, et cetera. I assume however that these can all be coded to numerical values in accordance with Srikant and Agrawal (1996). Does that clear things up?
Suggested by RamSharma: *Solution using kinship*: `require(kinship); model1 <- lmekin(yld ~1,  random = ~ 1|gen, varlist =list(covmat), data=mydata)`.
Brian, when you say "**why, in the previous question and answer, this issue of sample size does not figure into the calculation of the credible interval**", that is not true. The concentration of the Dirichlet distribution in that answer is determined by the sum of its parameters. You can check that this sum is equal to the sample size plus $k$. So, "automatically", when the sample size increases, you get a more concentrated posterior distribution that gives you shorter credible intervals.
Interesting that @EpiFunky is right, and I am pretty sure I am right too, but we gave such different answers!
[Wikipedia](http://en.wikipedia.org/wiki/Principle_of_indifference) $\\rightarrow$ ["Chapter IV. The Principle of Indifference"](http://books.google.co.uk/books?id=YmCvAAAAIAAJ&pg=PA41&dq=%22principle+of+indifference%22&redir_esc=y#v=onepage&q=%22principle%20of%20indifference%22&f=false).
Thanks very much for your input - I will read up thoroughly on this test.
@whuber - I was wondering why did you ask this question. I am curious, can you pls tell. The only reason I can think of is bootstrapping might not produce the same marginals. Am I thinking correctly.
The following document says "When an experiment is balanced, 
means and lsmeans  agree.  When data are unbalanced, however, there can be a large difference between a mean and an 
lsmean" http://www.public.iastate.edu/~dnett/S402/wlsmeans.pdf
The GIL is mostly irrelevant for machine learning. The reason is that all heavy computation will be in C extensions (as numpy/scipy) and these can run on multiple cores anyway.
One thing people have found with such *errors-in-variables* regressions is that the ratio of variances (of the y errors to the x errors) determines whether you should care much about errors in the x variables. If I correctly understand the ellipses in your picture to represent the joint error distributions, then those ratios are very large--perhaps 15:1 or more--indicating you could just as well apply ordinary least squares to your data.