tttjjj/elt_bench huggingface dataset:
@FraserOfSmeg Also, the exact conditions under which every experiment is performed are unique. So performing more tests will always increase confidence that theory makes reliable predictions about the future (ie is generalizable) or give opportunity for the theory to be modified to account for unpredicted results.
This answer clearly shows application of a standard test, but the approach is statistically defensive to a high degree. Testing marginal distributions is often irrelevant to later analyses. For example, in regression and other modelling, marginal normality is not even an assumption! The broader point is that the slight degree of non-normality here might or might not be notable, but that depends on **precisely** what the specific goals are here, which are nowhere made explicit in the original question. In fact all we have (so far) is a question title and a graph.
If the expected is actually zero and the observed is not zero, the chi-square value would be infinity. This is as it should be: you're observing something that according to the model is impossible, so it should automatically reject. Perhaps you should say more about how the expected values were obtained.
As a small note, the confidence intervals in se.fit are not on the response scale, but on the linear scale... you have to transform the upper and lower bounds through the inverse link function.

See for example http://www.stat.columbia.edu/~martin/W2024/R11.pdf, where it's done for logistic regression.
Oh, good note! Fitting to the Weibull distribution is egregiously wrong. It has continuous support and is never used to model counts but survival times. Negative binomial distributions would be a sort of equivalent two parameter distribution for modeling counts, which of course depends on the nature of the data-generating process (of which we have 0 information, as the problem is stated). Thanks for pointing that out to me.
Yes in both cases, in the same way that if I dig my garden with a hoe I bought from a hardware store or one I fashioned myself in a shed, I'm still gardening both times.
At present, this is more of a comment than an answer. Would you mind expanding it into more of an answer?
Problem is, if you look up "imply" in a dictionary you'll see both "suggest" and "necessitate."
it seems no one can answer the question, it is in the end a computational problem I assume since I cannot calculate the exponential function for very large numbers [even with range reduction]. Also I am limited by computational estimations of $k!$. I will leave it as it is and hope someone has an idea.
Clever. Maybe this is enough, and avoids the need to explain likelihood calculations.... Thanks.
@JuhoKokkala Ok, I will write a new question to address the exact question.
Summarize each histogram with a low-order truncation of its cumulant generating function and add those functions to obtain an Edgeworth-type expansion of the sum (http://en.wikipedia.org/wiki/Edgeworth_series ).  The computational effort is proportional to the total number of bars in all those histograms.
Usually $-2.615 \\times 10^{5}$. See e.g. https://en.wikipedia.org/wiki/Scientific_notation#E_notation (P.S. Did you copy this exactly? Are you sure it wasn't -2.615E-5 instead?)
Take a look at [this paper](http://www.siam.org/proceedings/datamining/2007/dm07_036Huang.pdf).  The authors develop a method for cluster temporal gene expression matrices in which rows are time series for gene expression.
+1, this is really nice. How did you make these figures?
Perhaps, you can use multiple external libraries whose code base does depend on each other. It is unlikely that independent libraries have the same errors which should eliminate some of your concerns.
@whuber, I'm not actually sure what's being asked here, \\@jank below, seems to interpret the Q as being about mutual causation, the thread I link talks about the $\\hat\\beta$s, other possibilities are: would the be an *association* both ways or would the *significance* be the same? I'm hoping for more info from the OP here.
Some of our false intuitions are due to heuristics and related concepts. See a lot of work by Kahnemann and Tversky. Some of these heuristics make sense, evolutionarily.
@whuber: starting with $Pr(X\\le x)$ i get: $$F_p(x)=Pr(X\\le x)=\\int_{-\\infty}^x dF_p(x)=\\int_{-\\infty}^xd[(1-p)F+pG](x)=(1-p)\\int_{-\\infty}^xdF(x)+p \\int_{-\\infty}^xdG(x) = (1-p)F+pG$$But is that a valid proof? No charactersitic function needed?
@user123276, If I claimed that I was taller than everyone in your family and you wanted to test that claim it would be enough to compare my height to that of the tallest member of your family.  We could compare my height to everyone in you family, but it would not change the conclusion relative to only comparing to the tallest.  On the other hand, if I can show that I am taller than some members of your family (but don't compare to the tallest), that does not really confirm my claim.  Same with the p-value, the test is based on all possible being less than alpha, so we just compare the largest
Hi Miguel! Thanks for your answer. I am not sure what you mean by "similarity ones". The similarity is in that sense not a feature but a measure how similar the features are. So I wouldnt know how to use that for a NB classifier. Also, could you maybe give a pointer to the feature selection weighting and chi^2 stuff? Maybe a paper or chapter in a book? Concerning your question: I want to evaluate them briefly to see if the features I design are generally capable of separating the classes and if its worth it, to build a special kernel for them.
Thanks @Macro I can see now why is expectation the average.
Minitab neglected to give you the most important output related to your question: The overall $F$ test (based on the overall $R^2$) for whether there is _any_ (assumed linear) association with _any_ hypothesized predictor and $Y$.  This is the only test you will see in this context that is perfectly multiplicity-adjusted.
(-1) Great explanation, but I would like to challenge an assertion. @Dave Kellen Could you please give a reference to where the idea that the TRUE model has to be in the set for BIC? I would like to investigate on this, since in [this book](http://books.google.es/books/about/Model_Selection_and_Multimodel_Inference.html?id=BQYR6js0CC8C&redir_esc=y) the authors give a convincing proof that this is not the case.
I think the relative time is what I meant as groupishness. Basically I want a way to measure how much time people spent in groups, but I suppose that depends on how I define my groups. Using my crude method (ie. distance to <= n) it looks like I can define my groups using my creteria and do that for each point in time like you said. Then I can evaluate either percentage of time each person was in a group over the course of our experiment or percentage of the experiment time a group existed. Do you think that will be a good approach?
There are cases in which uncorrelatedness does imply independence. See [here](http://en.wikipedia.org/wiki/Uncorrelated). To quote "...two jointly normally distributed random variables are independent if they are uncorrelated,although this does not hold for variables whose marginal distributions are normal and uncorrelated but whose joint distribution is not joint normal)"
Removing the co-linear predictors is necessary, not an option, because then as I said your model suffers from singularity. However asking whether adding new predictors improve the fit is rather a complicated matter which must be addressed in the light of model comparison/selection. You may see that having more predictors improves the fit, but it's not guaranteed that the observed improvement is brought about so called over-fitting. Having more predictors is equal to additional Occam's penalty (Occam's razor). As a rule of thumb, a good model is the one with the fewest parameters.
@Peter, Studentized residuals are less "distorted" by the OLS fitting algo and are closer to theoretical notion of ["errors"](http://en.wikipedia.org/wiki/Errors_and_residuals_in_statistics). They can be directly compared at different regions of the fit line, thence are better in decision if a point is an outlier.
thanks, good to know the Y was really a Y|P
One can't help noticing the high-leverage outliers at the left.  They contribute disproportionately to lowering the correlation coefficients.  Such ridiculously low scores likely represent people who blatantly blew off the test.  Because these scores can be recognized as anomalous in advance, if you intend to use these results for prediction, it would be valid to throw such scores out.
I don't think NIPS is a really a data mining conference, it is a machine learning and (decreasingly) computational neuroscience conference.
How many time points have you got?
I know that 10 fold cross-validation it gives a good estimate of accuracy but the question is does it give reliable estimate of variance in different models or does it underestimate variance?
If `SmallFirms` and `LargeFirms` are mutually exclusive and collectively exhaustive categories for the variable "size of the firm", then either one or the other must be the reference. _Tertium non datur_. Apparently, they're not MECE.
@Stijn yes ....
Your title and body are asking different questions. The *process* is called "estimation". An *estimated parameter* is called an "estimate", or a "parameter estimate". The formula or algorithm by which you compute it is called an "estimator". E.g. I might estimate a population mean, $\\mu$ by a sample mean, $\\bar X$. Then $\\bar X$ is called an estimator, and the value it takes on a given sample ($\\bar x = 113.4$, say) is the estimate of $\\mu$.
I psesume the very plot is drawn by some sort of multidimensional scaling or neural net. It doesn't bother me _how_ it was drawn, really. Does it, for you?
Could you confirm that "classification" is not meant as "cluster analysis" (which is called *classification* in French), that is do you really seek to apply a supervised method following your FA?
Could you please specify all the details of your regression? What are your dependent and independents variables? Furthermore, I guess you are using the Gini Index to measure the income disparity, but what is you poverty measure? The GDP per capita?
I miss the point of your last comment. Please ask clearly. Or use my above example data with 5 people and 2 nominal attributes and tell which individual with which you'd want me to compare and by what dis(similarity) measure.
Great suggestions!  By the way, if you order more than 10-20 copies of a book, publishers will often give you a *nice* discount.  At around 100 copies of a book, you're doing them a favor and doing their marketing for them.  When the book is revised, that's a lot of owners who may want to replace their edition.  :)
To answer your last-last question, you may find interesting this link: DOI 10.1007/s11192-011-0494-7 ‎-which is a recent article on an old discussion. If "negative results" tend to disappear from the public view, it follows by logical argument that most probably they have been _selected-out_
@Christina: vafisher is correct: in general you partition your data independently of its label (classification), so that it might add some variability to your calculations, reflecting possible future scenarios. The only reason I'd see for stratifying your data in this way is if the size of the classes is very unequal. You may also want to look at bootstrap validation.
I agree this is a good notation except for the usage of lower/upper to distinguish observation/random variable. But actually when writing statistics I never use the distinction with lower/upper, I prefer $x^{\\text{obs}}$ and $x$ (though this could cause problem when there is already an exponent). And for the sample variance, do you use $s^2_x$ ?
It's great to see you replying to questions, Jessica.  Welcome to our site!
Wow, starting right at me.  Thanks.
Ah, a Tukey mean-difference plot :)
Somebody will come up with an answer, but I suggest either writing down the equations yourself (with 2 variables it is not that hard) or looking up partitioned inverse and its implications for linear regressions.
There is no need to merge -- this is an example of "good duplicate"
Thank you very much for your reply Peter!
@NickCox, come on, it's just numbers! :)
I tried to run an r script in a Sage webinterface but it didn't work - it said that there was an error creating a png (no X.11 available or sth.)
Why not switch to logistic regression? To [quote Wikipedia's entry for linear discriminant analysis](http://en.wikipedia.org/wiki/Discriminant_analysis) (LDA), "Logistic regression and probit regression ... are preferable in applications where it is not reasonable to assume that the independent variables are normally distributed, which is a fundamental assumption of the LDA method."
You're right - I've missed this step. thanks a lot for your help!
I suppose I could post it. Will do so when back to my comp. Just wanted to first get intervals working where I can test them against a trusted routine.
Thank you for your answer, I hadn't it considered it from this angle, but it makes sense. Let me rephrase, to make sure I understand this point correctly: The model-comparison approach does not only allow you to test the effectiveness of the newly added predictor, but also whether the predictors that were in the model are affected by the addition.
Yes, I will have a look at it! Thx!!
Are `x1` and `x2` paired (and `y1` and `y2` also)?  You don't mention it but from the pattern of your sample data it seems likely.  You would use a different test if they are paired than if they aren't.
Just repeating your data is not acceptable. You don't give any eviews syntax (or equivalent). One of several crucial details here is whether you are trying to pool panels or not; there is no point anybody speculating because only you can make precise exactly what you are asking.
@whuber I am not sure what you are referring to.  I am referring to the confidence interval equation for y^ given in the linked Penn State tutorial that the OP gave.  That is for a confidence interval or else it would have an addition +1 under the sqaure root sign if it were for a prediction interval.  In either case x-xbar appears in the formula be it for confidence intervals or prediction interval.  My comment was to make the analogy to the OP for the case where his X is a dummy variable. Did I say something wrong?
I think you should add some detail. Perhaps you don't mean "statistic" in the first sentence. Or maybe you don't mean "reliability". "Split half reliability" and "alpha" (two different things) are usually applied to scales, not statistics. 

Please tell us what you are trying to do, in non-technical language.
Thanks, this was really helpful, I knew I was over-complicating it as the t-test for larger n approaches the normal.  So strictly speaking, even if n was 1000 the t-test should be used if SD not known a-priori?
@StéphaneLaurent: I like ["brand-name distributions"](http://www.stat.umn.edu/geyer/5102/notes/brand.pdf), for conveying that having been named doesn't in itself imply any special status for a distribution. "Known" distributions makes it sound like the rest may be out there somewhere waiting to be discovered, like the Loch-Ness monster or dark matter.
Jenn, FYI if you address chl by using the `@` symbol (like this: @chl), he'll receive a notification that you responded to him.
I find your question at the end unclear. What is it you want to obtain, exactly? What do the implementation of algorithms you mention give you?
Isn't it differential item functioning?
Can you give some more details as to the actual scenario? E.g. Whether you are using rejection sampling, how many parameters, etc?
The best way to graph this will depend on what assumptions you are willing to make. In particular, are you  willing to assume that sales are not seasonal?
I think you have to explain this a bit more, what exactly are you trying to do. An example or a sample of your data would be also nice.
Not wrong at all if you use the correct procedure. If you were to use the KS test assuming known parameter values, the critical values of the test statistic tend to a constant multiple of the correct ones as sample size increases, e.g. for a 5% significance level it's 78% of what it should be. So doing it wrong is very conservative. (Lilliefors discusses this in the paper.)
I'm not a specialist of time series and forecasting, but does it make sense to use a model which assumes uncorrelated errors in this case?
Two comments which might be helpful: (1) Did you see that a sketch of the proof of the desired result is given in the appendix of the paper? (2) Note that the distribution of $(X, X)$ is *singular* with respect to Lebesgue measure on $\\mathbb R^2$ and so your manipulations are not particularly rigorous. While the self-information result is true for (discrete) entropy, more care is needed in the continuous case.
If you want to understand what makes yield vary, wouldn't you want to sample from as many regions with different levels of potentially causal/explanatory factors as you can? If so, it seems to me like the tradeoff is between representativeness and the cost of sampling sufficiently from all such regions. Cost depends on other factors like data collection difficulty and statistical power needs, in addition to the kinds of factors you mentioned that determine the number of regions. Is there some other reason besides cost (time, energy, money, risk) to exclude regions? (To "sub-group," right?)
R lm works as expected, the problem is with your data, i.e. linear relationship is not apropriate in this case.
Thanks for the reply. Can you clarify? I'm not sure if you are specifying periods or saying to subtract. We wish to be able to predict at monthly and yearly intervals.
@ Staty Despair I see! Good luck with your further analyzes!
Did you take a look at the [tag:random-forest] tag? In particular, this question and the accepted answer may be of interest to you: [Obtaining knowledge from a random forest](http://stats.stackexchange.com/q/21152/930).
You need 700 packs; the chance of acquiring all 424 stickers then equals 90.0024%.  761 are needed to bring the chance to 95% and 898 for a 99% chance.  (On average, almost 560 packs are needed to complete the set.  It is unlikely (less than once chance in a thousand) that fewer than 352 would be needed.)
Glad to help, Tom. If you found my answer helpful, I'd appreciate it if you would mark the answer as "accepted" -- just click the green check next to my answer. And I've actually written out the R code to replicate the rat cancer example, if you would find that helpful.
Cross posted: http://stackoverflow.com/questions/13870427/individual-level-parameter-estimates-from-ordered-choice-regressions
Thank you for the detailed answer, I'm pretty new at this. The data is the same except the correct answers are transformed as you wrote. The two prediction models are different, and I want to assess which is better. For one of the models I only have MSE and one pair of values for sensitivity and precision (rather than a complete ROC curve). The other model is at my disposal and I can measure it however I want (MSE, ROC curve etc').
Almost, but not quite. It's true that you build up the model by adding the "best" variable of the remainder to those that you already have. Once a variable goes into the model, it remains there. However the addition of each new variable involves inverting the X'X - where X is the design matrix at that stage. When the columns of X become linearly dependent (or nearly so), then inversion of that matrix will cause the algorithm to fail.
@Franco: Frank Harrell's own Regression Modelling Strategies - [book](http://www.amazon.com/Regression-Modeling-Strategies-Applications-Statistics/dp/1441929185/), [website](http://biostat.mc.vanderbilt.edu/wiki/Main/RmS), [R package](http://cran.r-project.org/web/packages/rms/index.html) - should get you going. A brief discussion of some of the advantages of regression splines is [here](http://stats.stackexchange.com/questions/5015/what-are-the-advantages-disadvantages-of-using-splines-smoothed-splines-and); but you're right of course that polynomials are an alternative.
@Glen_b Well thats what I am trying to determine. What is the resulting PDF of this?
You can easily recode the numeric values to the nominal scale, but of course this loses information. It's also possible to keep the two types of measurement, and write a likelihood function that incorporates both kinds of information (though it may be risky to assume they both represent subpopulations with the same underlying distribution).
You could add the 'homework' tag to help people categorise the question and answer it in an appropriate way.
Thanks a lot for the clarification/explanation! Yes, my confusion was mainly about the interaction part. Thanks again.
The square of an exponential r.v. is [Weibull with shape parameter $1/2$](http://en.wikipedia.org/wiki/Weibull_distribution#Related_distributions). Its convolution with itself is not analytically tractable, but the convolutions can be done numerically. But I expect you knew this.
Using the "continuous age is available" as a predictor will result in major problems of interpretation and inference.  The problem as originally specified is an ideal candidate for multiple imputation.  For those subjects for whom continuous age is known you also know the bins.  That allows you to develop a multiple imputation model for continuous age given the bin.  Then you can form a model with only continuous age (90% real, 10% imputed).
@rolando2 Thank you for your feedback. There is a comment about `bw.nrd0`, which is the bandwidth used by the command `density()`. I have added a link to the description of this command. I will try to soften my words also.
Does this approach make sense?
hmmmm you think it would do any good if we had data from previous year around the same time period as current??? To be frank I feel there is a lot of uncertainity involved and we are making a lot of assumptions here
Did you try performing a simple linear regression? If so, what was the $R^2$?
(+1) Nice answer, @Glen_b!
It might be of interest to note that when the window size is odd, it is possible (although, I admit, unusual) for two overlapping windows to be *uncorrelated.* For instance, with a window of size $89$ and an AR-1 series with serial correlation around $-0.985737,$ successive averages will be *uncorrelated* (but still highly dependent). This indicates that it would be better not to substitute lack of correlation for independence in reasoning about this question.
Just out of curiousity, why did you choose to do a one-sided test? (Too late to change your mind now..)
I'm sure Vapnik would not say that again: 2006 deep learning came up and showed that there can be enormuous benefits.
Looks like it's referring to the response distribution value that you can enter at the top - i.e. the percentage of subjects you expect to be affected.
@PeterEllis: My argument is based on a fixed amount of resources to do the survey/census. If you focus on, say, an appropriate 10% of your population you'd in theory have roughly 10x the resources to attempt to gain a higher response rate. Perhaps I'm being naive.
Thanks. Unfortunately, few of my data are numeric, but this is a useful phenomenon to know about. Incidentally, there was a nice programme about this on BBC Radio 4 a few years ago - looks like it's still available in RealAudio: http://www.bbc.co.uk/radio4/science/further5.shtml
Thanks for the comments. I tried to upvote your answer but I don't have enough reputation right now.
But if I pay for them, I will get them.
Suppose the exact variable values are $x_i$, the exact output value is $y$, observed variable values are $\\hat{x}_i$ and the observed output value is $\\hat{y}$. Is it true that in your model $y = \\sum_i w_i x_i$ (i.e. the exact output is a linear function of the exact variable values)? Then, if you had access to $x_i$ and $\\hat{y}$, this would be standard regression. But the issue is that you don't have $x_i$, but only $\\hat{x}_i$, so you want to build a model of the form $\\hat{y} = \\sum w_i \\hat{x}_i$, even though the true model is $y = \\sum_i w_i x_i$?