tttjjj/elt_bench huggingface dataset:
Standard error for interaction terms doesn't mean what you're suggesting. See here: http://www.unc.edu/~enorton/NortonWangAi.pdf
This is true, and worth pointing out, but it may also be worth mentioning that in this case the $p$-value should be really thought of **only** as an index of signal strength -- such small $p$-values (sometimes even if corrected for multiple comparisons) are so tiny that the probability that the NSA broke in and tampered with your data (and then brainwashed you so you can't remember) is far, far, higher than the nominal $p$-value.
Can you say more about what was wrong / inadequate about the correspondence analysis?
Hi Charles and thanks for your message.  Do you have experience with using the post-model calibration methods you mention, Platt or isotonic regression?  I'm not very familiar with either method and would like to know how to implement and test.
@andrea. Thanks for this interesting link. My main problem is how to get the figure with ggplot directly from individual data (in the example above data0) with ggplot. For me it is more technical/programming problem (for stackoverflow) than a statistical but the administrator seems to have another opinion.
For more than 100-200 users, I would increase `threshold` to about $2.5$ to avoid false positives.  *E.g.*, try the code with `n.users <- 500`, `n.outliers <- 100` (that's a large proportion!), and `threshold <- 2.5`.
"*a slightly reworded problem from a book I'm using for self-study*" ... which sounds like it might still count as worthy of the `self-study` tag. ([take a look](http://stats.stackexchange.com/tags/self-study/info) and decide for yourself)
I confess to being mystified by this, despite struggling several times to make sense of it: is there a question here?  "So bad" is vague and has no referent.  What exactly are you seeking to understand or find out?
1. you are almost right about the application of this test. 2. Regarding Mahalanobis distance, I don't understand how can it be used to assess repeatability. Do you suggest computing covariance matrix for all the points in all the methods TOGETHER and then comparing the methods by sampling MD using that matrix?  3. Downstream application is indeed a valuable option, however it will not reduce the dimensionality t
@PeterFlom  Thank you Peter for this question---I should have clarified earlier.  I just expanded the initial post with: "Suppose the goal is that an event happened in the Central Time Zone that did not happen in the Eastern Time Zone.  My goal is to evaluate the impact of that event on say the county's average income, crime rate, and death rate."  I hope that clarifies, but please let me know whether I should clarify further.
A suggestion: could you give an attempt at an interpretation yourself? And maybe specify what you what to test for? A quick hint for the former, is to look at the definition of the Cox model, identify the linear predictor and try to study the proportional hazards property of the Cox model.
@whuber The simulation predicts multiple collisions, but since an object is to be removed once a collision occurs this should not be possible. So if A and B have probability x1 and A and C have probability x2 then I test each probability (random number) if A results in being in a collision at any point it is then removed and further collisions aren't calculated for A. I know this is an issue since it decreases the likelihood of collisions with objects further down the list being simulated. I plan on fixing this by making a list of all objects that collide with A and then selecting 1 randomly.
This question appears to be off-topic because it was cross-posted on stats.SE and answered there.
Since there are large sample sizes, z-test is an alternative option.
@mbq - whats wrong with the question?
While I think this is an attempt to respond to the question and contains several suggestions in response to issues raised in the question, I think this answer could be improved in several ways; in particular, if the OP responds to your questions, it might help if you could expand on your responses.
BTW, I just took the log of d.  d can include zeros, but the logistic function maps that to zero, which is what I wanted.
This question might also be of interest: [How to cluster longitudinal variables](http://stats.stackexchange.com/questions/17772/how-to-cluster-longitudinal-variables)
I did a moderator flag to do so. Hopefully someone with appropriate rep can move it.
Just to clarify - are you testing for a generic lack of, e.g., Normality, or for the presence of a discontinuity at a pre-specified point?  Your example is of the latter, but of course any goodness-of-fit test, e.g., Anderson-Darling or Shapiro-Wilk for Normality will serve, although with a highly specific alternative you could construct more powerful tests.  Also, in your graph above you evidently have a sample of thousands; would this be typical too?
Apparently there is no simple answer :) http://www.amazon.com/Nonlinear-Series-Analysis-Holger-Kantz/dp/0521529026
Sound statistics with small data sets are always difficult. So you'd recommend to stay with a Poisson GLM, possibly transforming the predictors if necessary? I've forgot to supply some data: x <- c(10.4,6.6,7.3,11,14.6,13.2,10.4,8.4,1.8); y <- c(0,1,10,34,52,58,8,19,4)
http://cran.r-project.org/web/packages/BMA/index.html should help, i think.
I think you do: the joke is not at all apparent.
I think that rather than a hypothesis testing problem, this is an estimation problem, unless you have an explicit hypothesis about the variance of the random variable (not to be confounded with the sample variance).
It is relatively easy to make that model account for autocorrelation. One option is to use `gamm()` and employ the `correlation` argument of the `lme()` function (of package **nlme**. Another is to follow the example in `?magic` (also in **mgcv**) whereby you make some guesses as to the complexities of the smooths, fit model, estimate the appropriate covariance matrix and use it to update the model fit.
Paying attention to how wide is the `step width` exposes that one actually thinks his scales are interval, not ordinal. Ordinal scales do not have a defined step width, albeit they do differ in respect to the number of levels (=categories, =points): some are more grained and some are less.
Cardinal, I've actually got a lot of data that looks like this too. We were trying to collect fully paired data, but due to technical problems or bad luck, some samples measurements under A or B are sometimes spoiled. 

The two obvious--but unsatisfying--solutions are to 1) throw out all incomplete pairs and do a paired t-test, or 2) ignore the pairing and do an unpaired t-test across all the data. I think the poster is asking for a way to leverage the pairing where it exists (for your reason #1 and #2), while salvaging whatever he can from the other, unpaired, data points.
**Hint**: Use symmetry. Bound just(!) the density part. Recognize a very simple and familiar integrand. Declare success.
Check out the material on hierarchical regression and causality at http://stats.stackexchange.com/questions/10020/comparing-importance-of-different-sets-of-predictors
Showing some work, ideas, and motivation is always appreciated. Without giving much of an indication of what you've thought about, what you know and what you're unsure about, its difficult to provide an answer with any real long-term pedagogical value. Cheers.
The maximum likelihood estimator *is* OLS. As they are exactly the same, they will have the same asymptotic properties.
Hi mtal, I had already seen the video, and tried to look through bokeh's code, and I also bought the GoG book, and tried to put it all toghether in my mind, but still I would have liked someone with more experience than me in this field to give me a few pointers on which direction to start the coding process.
What could be the possible features for this? I've extracted about 10 features and trying to use SVM, will this really help?
(+1) Nice response. I'm not so surprised to see that Rothman & Greenland's textbook is again a good reference to keep on our desk, always within easy reach.
Actually we could do better than this ... using AB testing we could pick to display half of the -1 voted question on the site as 0 and half as -1 ... and see if either of the groups is more likely to be upvoted! Ingenious.
From what you've said it certainly sounds like you've no reason to categorize anything. Of course interpolating or extrapolating into regions where you've no data is not free of risk; the alternatives are collecting more data or not using the model to make predictions.
Oops the above one was for you, @EricPeterson
Distance cannot be negative by definition. Euclidean or Mahalanobis d, in particularly, are computed out of their squared versions, as the root, so they cannot be negative.
@NickSabbe, thanks for this reference. It is very useful. In the context of GWAS, and probably also in other contexts with a huge number of correlated predictors, I heard [Sylvia Richardson](https://www1.imperial.ac.uk/medicine/people/sylvia.richardson/) recommend Bayesian model selection based on e.g. some comparisons with stability selection. The MCMC computations were really demanding, though.
@user11852 The paper you linked to makes a distinction between deep neural networks and deep belief networks.  Deep belief networks have a undirected connections between the top two layers, like in an RBM.
@JacobMick Say I want to compute the standard deviation of the five-period average of my observations. I could take (say) 100 observations and split it into 20 blocks of 5 observations. Alternately I could use 96 blocks, each of which shares 4 observations with the block on either side (a rolling window). The sampling variance is reduced by a factor of 2/3 in the latter case, so there's a definite advantage to using rolling windows to compute variance. I wonder if the same can be said for linear regression, or if there is no benefit in this case.
"Have to" isn't really applicable. You can do anything you like if you're prepared to live with the consequences. The assumptions of a normal theory test aren't satisfied; but if they're not close enough for your purposes you can use a similar analysis either with more suitable assumptions, whether you evaluate things algebraically or via simulation, or you could use resampling techniques, or as you suggest, nonparametric techniques. The major considerations relate to what questions you need to answer and the tradeoff between power and ease/simplicity.
When you say 'constant variance', do you mean 'known variance'? For non-constant variance, unknown variance?
It's mature, well supported, and a standard within certain scientific communities (popular in our AI department, for example)
Cool! I upvoted for you. However, after reading your explanation, it looks like I would like to actually compare the following two: (1) `lm(residuals(lm(y~x1))~x2)`   vs. (2) `lm(residuals(lm(y~x1))~residuals(lm(y~x2)))` ... Any meaningful observations can I make from this comparison? Thank you!
@ whuber: I read the similar post b4 posting this question. The context of my question is different & also mine includes two different questions the 2nd of which has not been asked in that post. Kindly please pay more attention to the context and specs of the question b4 closing it, TQ.
What is the story you are going to tell?
If you wish to explain a greater skill in Math than in English then $M-E$ looks good, but if you are interested in overall skill, why not $M+E$?
@Gilead I did not know about Taleb but I read a book by Jean Pierre Dupuy in 2004  http://fr.wikipedia.org/wiki/Jean-Pierre_Dupuy, the book is called "Pour un catastrophisme éclairé" and if I reamember, it talks exactly about that. Jean Pierre Dupuy is Teaching at Stanford and at Polytechnic but if I think he mentionned Ivan Illich while talking about that. Does Taleb mentionned sources of inspirations?
When studying causality, you have to first establish it. You have an endogeneity question here: whether the consumption impacts government spending or not? I'm assuming this is some kind of course work. You better consult with your macro professor on these things.
search for recommender systems. this looks like a problem that can be adopted to that. start here http://en.wikipedia.org/wiki/Recommender_system
I think @cardinal is correct Dilip.  Most of your reply looks good, but this part seems mistaken: "However, the z's from the studies will be positive and negative respectively."  This presumes the z's would be calculated as if the studies performed *one* -tailed tests.  With a correct two-tailed test, the z's will both be positive, not "positive and negative respectively" as you claim.  There's still some thinking to do here: although Stouffer's method will conclude there is a significant difference, it cannot tell us in what direction!
Looks like you got two answers that hit pretty much all the same points posted within a few seconds of each other. How's that for independent confirmation?
@nominator There are faster ways, including inverting the CDF and rejection sampling. Wikipedia remarks there is no universally fast way, but specialized methods have been developed for various ranges of the parameter. (Note that a chi-squared variate is a scaled Gamma variate.)  For some of these methods visit the thread at http://stats.stackexchange.com/questions/7969.
BIC gives you a rudimentary ranking of the models under comparison, but it does not tell you how (more) likely one model is compared with the others. The posterior distribution over the models (states) contains in addition information about the parameters of each of the models under comparison. You can condition on a model (state) or average across models to run predictive evaluations. This is much richer than using a single-dimensional approximate criterion like BIC (or any other IC)...
Could you explain how "merging" the covariances will help you create a mixture distribution?
Hemmo, I don't know that I'd necessarily go so far as to especially advocate that as a method of model selection, but if you are looking for a basis for choosing, it's typically more important to get that part right. If you have the ability to do sample splitting/cross validation, you might consider using it in those circumstances.
If you want something other than a generic, approximate solution then I think you need to provide more details on the exact form of $f(x)$, $f(y)$ and $f(x,y)$.
@whuber Or simply $m/M$ which is well-defined and in $[0,1]$.
Hi @PeterFlom Well ... it is a cross-posting of the exact same question, by the same author, to multiple SE sites. My understanding was that that is discouraged.
By the way, I miscalculated. The variance is $\\frac{1}{3}$ so the standard deviation is $\\sqrt{\\frac13}$.
What's the purpose of the experiment?  The answer to that determines the answers to all your questions.
Yes I understand that A, B, and C can be achieved by first splitting into A&B vs. C and then splitting A from B.  My point was indeed that a given algorithm might not choose that particular sequence.
Nice answer. You have not mentionned that using either $t$ or $z$ is strictly equivalent because it is obvious. I would add it because (oddly) it is not clear for everyone, as shown by the upvotes to the first answer.
Thanks - this is a nice post! Maybe not directly applicable to my intraday setting, but very illustrative.
It doesn't have a precise meaning in general, but for specific problems there is a precise meaning - otherwise stratified sampling wouldn't be used so much.  Polling is definitely one of these cases.  For example you would want a good age distribution within your sample.  If say there was nobody in a "random sample" aged over 50 then you would be suspicious of any polling result based on this sample.
@chl: The first paragraph shows the Incidence Rates of 2 diseases (cases per million per year) being compared
The second paragraph gives the actual number of cases of 2 diseases being compared
Nice drive by down vote. Please explain, why the down vote?
@deinst I was not aware of the self-normalization procedure and its pitfalls so thank you for this! In any case, I think the issues may be relevant to the properties of my IS scheme so I would like to explore this idea some more if any of you have ideas.
@AndreasRybicki, if this is what you needed to know, you may want to accept ocram's answer by clicking on the check mark to the left of it.
Thank you! Now I feel much more confident about splitting the data set.
@NickCox I tried to address your questions. Does my edit make things more clear?
*Any* nondegenerate binormal distribution would serve that purpose, Dave. The simplest example would be a pair of independent standard normal variates.
Anything that is seasonally adjusted should successfully adjust for this. If you were actually doing a histogram with frequency density you could have bar width proportional to length of month and height proportional to count per day.
Our current policy is to not weight, since I, like you, decided that the weights were rather arbitrary and were probably no better than the unweighted results.  When we sample, we target a specific distribution using each segment's proportional size, but this is also using a relatively arbitrary target distribution, so we may need to change this policy as well.
I need more steps to proceed this problem
Thanks for your detailed answer! I don't have tbegin (the time of the previous visit), but I guess I can transform the data myself. I was kinda hoping this was a solved problem, but it doesn't seem too hard anyways.
+1. This is a very clever example that hadn't occurred to me.
This answer looks mostly absurd and out of the question's context.
Hmm I don't think thats correct. If K is the linear transformation taking X to Z, and P is the linear transformation taking Z to Y  (PCA finds P). In linear algebra's terms, we write K*X = Z (K could be normalization or standardization) , and P*Z = Y (P is found through PCA). My question was, what is M, the transformation that takes X to Y, written M*X = Y. M is found by simple linear algebra! Someone on another forum helped me. Y = P*Z = P*(K*X) = (P*K)*X (matrix multiplication is associative) Hence
Y = M*X = (P*K)*X, which implies M*X*X^-1 = P*K*X*X^-1 so
M = P*K !
@semibruin Thank you, but that question doesn't seem to be very relevant.
I am not suggesting that the model is wrong as such, but that it is unhelpful in that it is difficult to get accurate estimates of the parameters: different parameters would produce similar results, and you cannot easily get estimates of $N_e$ from $N_o$ since $N_e$ is an input to the calculation of itself.
Note that the history on wikipedia probably isn't complete.  You will need to do more research yourself. In some reference - which I unfortunately did not keep - some earlier resources were mentioned.
In my Variogram Analysis I am analyzing population points. So I assume the unit would be people.
@ttnphns I didn't notice the previous question, my bad! And now it seems there's some confusion about what you are after, wxa. Do you want to use a *measurement model* or are you just interested in modeling multivariate relationships between two blocks of data? In the latter case, one way to account for the asymmetrical role played by each block (6 predictors on one side, 6 response variables on the other side) is to use *redundancy analysis* or *PLS-2 regression*. (I used to think OP means either "original post" or "original poster", depending on the context)
@Aniko: I've updated the question with example output.
welcome to the 10K club!
@Ariel can you add as answer?
From the perspective of the 2x9 matrix, you have 15k samples of 9-dimensional data.  Since you are only learning two principle directions, there is not much difference between 1000 vs. 15000 "samples".  Yes, sampling without replacement is technically not bootstrapping, but, practically speaking (considering your problem), these are very small differences.
It seems to me that the error will depend on the type of classifier used. Anyway you can always try to predict at different sample rate and fix a threshold where you think the error introduced is satisfactory.
@Vilx-, can you provide the latvian term in the comments? I am Lithuanian, so I might guess better. Also in what context that term is used? Maybe there is some data on which it is calculated? It would be easier to  guess precisely what it is.
@PeterFlom, the data is derived from a neurophysiological experiment and needs to be normalize/z-scored prior to any further analysis. I was wondering what is the easiest way to compare the three groups given the z-scores.
Thank you very much for taking the time to  answer my questions, There are terms that I am not familiar with and need to read more like PCA projection and MSC without fixed reference. Also, I think binom.wilson is an R function?  I  will have to find similar one in RapidMiner. 
Also, When you mentioned  2 sample test for equality of proportions, do you mean 1 sample from the training and another 1 sample from the test set? I will read the link that you gave. Thank you very much for that too :D
Is it just spitting out the lower and upper limits based on the range of the data, or doing something else?
How is it possible that your cost is not changing? The only thing I can think of is that your lambda values aren't actually different. Maybe posting code for what you have done will help to see if your situation can be re-created.
Can you explain how you would normally use AIC to determine the number of lags? This question isn't specific enough for me to know how to answer.
(+1) Very nice answer. I like the example. Is there a ready interpretation that you know of to motivate the use of the Euclidean distance that you've given? I also think it might be interesting to point out in this context that the ROC curve is essentially obtained by making a post hoc modification of the intercept estimate of the logistic model.
No doubt someone will write a good answer, but in the meantime, there's an outline of the calculation [here](http://wise.cgu.edu/powermod/computing.asp). Keep in mind that they're doing a one tailed test calculation (hence the difference with $\\alpha$ vs $\\alpha/2$), and that $Z_{1-\\beta}=-Z_\\beta$, and their $d$ corresponds to $\\delta/\\sigma$.
Great answer and brought up issues I hadn't thought about.  Perhaps I'll use your equation and comments to stress how important it is that the sample size grows to reflect increase in factors.
It depends on what you mean by "best" and on the nature of your data. For example, if $y$ is measured with additive error then really $y = \\beta x^\\gamma + \\varepsilon$, which is nonlinear, but if $y$ is measured with *multiplicative* error then $\\log(y) = \\log(\\beta) + \\gamma \\log(x) + \\delta$ is actually a linear model. *The two models are not the same*: typically they will produce (slightly) different estimates of $\\gamma$ and different standard errors. If also $x$ is measured with error, both models get substantially more complicated and yield yet two more estimates.
Thanks a lot for your help, I've been googling for hours, and reading extensively 'elements of statistical learning', to no avail. Thanks a lot.. One more thing while we're at it, if you don't mind - are the trees weighted? or do I just average them out to get my prediction?
Thanks for your answer! I don't know if i made clear that i already set the y-axis logarithmically. But i also realize that when i transform the actual data to log10 _before_ plotting, only what i would consider "real outliers" is displayed as such. Which makes sense, as all values are now single-digit and may be covered by the IQR.
You need to estimate the covariances. Also, as @Glen_b points out, you need to divide by 3 for the mean.
How about bootstrapping the mean, obtaining bootstrap confidence intervals for the mean (http://en.wikipedia.org/wiki/Bootstrapping_%28statistics%29#Methods_for_bootstrap_confidence_intervals) and multiplying the estimates and the interval boundaries by the total population size?