tttjjj/elt_bench huggingface dataset:
I'm sorry for your confusion. My teacher suggested me to do one-sample t-test for this. I didn't know about the test at all, so, your answer (plus your explanation added later) was very helpful. I really appreciate it. I'll generate a new question with more details as you suggested. Thank you!
@mpiktas - This expectation actually does *exist* but it is infinite.  A better example is $g(x)=x$ for the Cauchy distribution.  This expectation depends on how the lower and upper limits of integration tend to infinity.
Because the entire set of possible tuples of $(x_1, x_2,x_3,x_4,x_5)$ contains only $5^3\\times 4^2 = 2000$ elements, why not just do an exhaustive analysis?  The concept of a Sobol sequence does not seem to apply to discrete sets, anyway.
I think that if the ellipsoid is of the form $(x-\\bar{x})^TS^{-1}(x-\\bar{x})=g^2$, then g should be given as radius.
Thansk everyone - I only have one group - I'm trying to calculate Orwin's (1983) modification of Rosenthal's Failsafe N and it seems like this is only possible using Cohen's d. Any suggestions?
@Wayne, I'm aware of the paper, thanks. But Gelman and Stern are arguing that estimates of a mean, say, between two samples may not be statistically different even though one of the estimates is significant while the other is not. I'm looking at a different issue: Why no confidence intervals around a p-value for a particular sample?
Not a full answer, but re: "Is this something to do with 'one-tail' and 'two-tail' testing?" No. That involves the direction of the relationship, not its magnitude.
Is your response variable (transfer) 0/1? Usually odds ratios are for a dichotomous response; I'm not sure I see how they would function in this situation. It's worth noting that model betas are a measure of effect size, just not a standardized one. Also, can you include a link to the other CV question that you think is related?
Am I correct that the closest known design is the split plot ?
Thanks, realized this yesterday evening myself. The problem boils down to a quite simple one in the end.
For an interesting practical application of this ambiguity, please see http://stats.stackexchange.com/questions/34396.  (BTW, the sign reversal *was* noticed: see the first comment to this question.)
Perfect, that looks great.
The easy way to find out is to feed it some simple datasets having ties!
Quota sampling is distinct from stratified sampling in precisely this respect: unlike selection in a stratified sample, the lack of randomness in quota sampling means we cannot assure lack of bias.  So: maybe you can't erect CI's for estimates from quota sampling, but you *can* establish intervals for the bias.  Unfortunately, except when the sample is a substantial proportion of the population, the bias extends from almost -100% to almost +100%.  This is why quota samples have long been discredited ("Dewey defeats Truman", etc.).  *Caveat emptor.*
The "extrapolation" in the title says it all: *you are making up the answer arbitrarily* when you estimate extreme values from smaller datasets. All such estimates rely on assumptions about the shape of the tail; there is no getting around that. Methods like those you have discussed are just ways of hiding this fact, which explains why they do not work. Your only hope is to learn something about the tails either from theory or from other related datasets and incorporate that knowledge in your modeling.
I am not sure thus actually answers the original question, as it was about univariate vs. multivariate anomaly detection.
Forgive me if this forum's users don't use R, but I am trying to produce a scatterplot of the multiple regression results for this data with two IVs (one categorical and one continuous) and one DV...is R even capable of doing that?
No, I think I'm all clear on it now :)
Also, for your first question: The recoding was done like this; 0 damages (0), 1-5 (1), 5-10 (2), 11-15 (3), no more levels were added as there were no need (no single case had more than 15 damages). Where the signs in the parantheses are the values assigned to the levels.
So yes, I know that using the outcome of Step 1 (finding the split) may be incorrect and thus would throw off the result of Step 2.  I'm cool with that because I can test the result of Step 2 experimentally and see if my assumptions were correct or not.  But I still don't know the 'correct' way of doing either of these steps.  I was hoping there was a real statistical method for doing this; as opposed to just making one up myself that works but isn't as robust as it could be.
This is Creative Commons but does not allow derivates... :( That way, I cannot use the bits in my own course material, just the bits I need them to know...
@image_doctor: I might be able to get that. Do you mean that it would be good, because vibration/noise is relevant to the problem domain? In that case, couldn't I summarize it like the other data? Or is it inherently about the information processing being in real time? I would like to be able to predict faults well in advance, like a day or (preferably) more. I don't know if it is feasible with the domain and the data. But this timescale doesn't suggest to me, that real time processing really helps. If you mean that the real time is significant, have I misunderstood something?
Yes, this is a much clearer answer than mine! +1
Hint: when you know $a_2$ and $a_1,$ you also know $a_2-a_1$ via subtraction. What information does that give you about $b$?
Do you mean ARIMA rather than X-12-ARIMA? The latter is a decomposition method not a forecasting method.
There's really no significant evidence that the abundance varies significantly with distance!  Because most of the data are obtained at close distances, you should *expect* both the largest abundance (and the smallest!) to be at short distances: which they are.  The presence of the outlying large abundance at a short distance, although it creates the *appearance* of a trend, therefore could be entirely accidental. Visualize this in `R` with a plot like `plot(d, sqrt(a), cex=(1 + sqrt(a))/3, pch=19)` where `d` is the [Distancia] vector and `a` is the [Abundancia] vector.
@PhD I was thinking a good plot would be to take the year on year change in participation vs year. Then you should see a large spike when tool B was introduced if it helped. There may be some signal detection algorithm that could help you get a "substantive" number to back up your claims. If you find one please post your own answer.
I just wanted to add that I've used the package and it has worked very well for me (but that was on pretty simple models).
What exactly do you mean by "multivariate... regression"? As the tag wiki excerpt notes (mouseover the tag `[multivariate-regression]` to see), it usually stands for a regression model where there is >1 *response* variable, not necessarily >1 *predictor* variable (although there may be that as well). Is that how you are using the term, or do you mean a model w/ >1 predictor variable but only 1 response variable?
Is this some consortium or resource for generally connecting people needing expertise with those that have it in their particular area? Or, is it simply a private consultancy offering services related to data handling? The website lacks much in the way of specifics. I could not locate much in the way of specific areas of expertise, though some very broad categories were listed. It is hard to tell who one would be interacting with.
Re the "does not apply" option: Are you suggesting you would impute values where the respondents say they are not even applicable?  For instance, would you really try to use the female responses to a health questionnaire to impute the numbers of pregnancies that male respondents have had?
After some research I would like to recommend readers who are concern by the same issue to have a look at Data Analysis Using Regression and Multilevel (Gelman A and Hill J) and Applied Survey Data Analysis (
Steven G. Heeringa, Brady T. West, Patricia A. Berglund ). I'm not an expert but I feel both books are good. Maybe the first is a bit more intuitive and general the second on the other hand the second is maybe more difficult but reall really related to surveys...
Optimal scaling procedures require categorical data coded as positive integers. There is `Discretize` button in the procedure, to help you recode continuous variables automatically in a useful way, or you can recode manually.
Yeah, it's a degenerate distribution at 0. Thanks for the suggestion, Michael.
Also, would you be willing to make that a full answer, Procrastinator?
Nope, I use R almost completely. Are there some packages that your professor or other economists use a lot? I don't use any econ specific packages. I use plyr, matrix, and many other packages, but none are designed uniquely for economists.
Then it will be biased another way (toward 0's).
I don't have his dataset, so I cannot estimate his model. I have variables in my dataset he didn't use, and he used variables I don't have exactly. There is another dataset that I could test both his model and my model against, however, and I'm wondering how I could go about doing this.
The search function will find you answers related to this, like [this](http://stats.stackexchange.com/questions/48317/kolmogorov-smirnov-with-discrete-data-what-is-proper-use-of-dgofks-test-in-r) or [this](http://stats.stackexchange.com/questions/13326/can-i-use-kolmogorov-smirnov-to-compare-two-empirical-distributions). There are other possibilities, but it's really a new question.
I only differ with IrishStat with semantics.  If a series over time has no time structure people sometimes refer to it as white noise.  But it is still deifned to be a time series even though it is useless at predicting future values in the sequence.
@Glen_b: Tricky to cross-validate an ill-defined procedure though - in each fold you'd need to repeat the process of looking at diagnostics, thinking up another transformation when you didn't like them, trying that, & so on.
Of course you can fit the combined model in R and test interactions. Whether you need to take account of multiple testing would depend on whether you did other tests and your overall tolerance for type I error; if you could tolerate getting say 5% of type I errors when the null was true, you needn't make any adjustment. If for some reason you need strict control of your overall type I error rate, you might want to worry if you do more than one test. Note that here, that F test is a single test; if that's what you're testing, it already has the overall type I error rate that you have specified.
[This](http://linkage.rockefeller.edu/pawe3d/help/Beta-distribution.html) shows how to find the parameters of the beta-distribution in terms of the mean and variance
That could be.... I don't think it's what's happening here, based on question, but it could be. The question has very little detail.
I am interested in knowing whether we can get an estimate of the validity coefficient with the help of CFA ?
+1 Nice answer: clear, to the point, insightful, and authoritative.
If the five other homes are similar in size, lot, location, # of bedrooms, do you need a regression?  Why not just take the average price as @EmreA suggests?  What are your explanatory variables anyway?
*Now* we have a clear, answerable question!  (+1).  I look forward to reading the solutions people have to offer.  (Automatic labeling of plots is notoriously tricky and manual labeling is a pain, so there is a real problem to be addressed here.)
The question is not about OLS, however: it is about the "total least squares estimator." This based on an errors-in-variables model. For scalars it is indeed true that the coefficients are reciprocals.  This relationship does not extend to higher dimensions, though (which is obvious just from looking at the dimensions of the matrices involved).
As I state in the question, no, and then I ask if there are any general cases when we would not expect the two to outperform one another.
As I pointed out, names do not usually determine duplicates: any city of a moderate size will have distinct streets with the same name. Using telephone numbers assures you will only count a subset of streets: for instance, many main highways might not have any registered addresses; no limited-access highway will show up that way. Whether these biases in your recommendation are important in any circumstance will depend on the application, but it's important to understand they are there.
Thank you very much for bringing this to my attention and for the reference which I will study carefully.
Combining P-values usually involves the use of one-tailed P-values and so p=0.5 is what you get when the data maximally agree with the null hypothesis. ("Most insignificant" is an awful description.)
@GJ Kerns: l is fixed but arbitrary, question updated
1. The standardized regression coefficients were not the same (0.91 and 1.02) for the model with and without an intercept). 2. The intercept was 9.5 (s.e. 1.7, p<0.001). 3. As far as I know, many papers suggest not to suppress the intercept even if the intercept was not significant from zero.
Why exactly do you need a formal distribution? If you have enough data, I think you could use your data itself, and some sort of method like kernel density estimation to get 95% of your data in a 3 dimensional shape.
@Lee, glad to help. We'd all like to be more certain, but I really do think that you're fine here. Incidentally, if this is what you needed, you may want to accept the answer by clicking the check mark next to it.
Univariate boxplots are useful for spotting univariate outliers. But they can completely miss multivariate outliers.

The regression idea is ok if I had a Y and a bunch of X variables. But as I said in the question, there is no Y so regression is inappropriate.
You're going to have to provide a bit more information, and perhaps output. But two guesses: Missing data? If some people were missing scores on the between subjects factor, you'll have a different sample. OR: The means are residualized (I forget what SPSS calls them), so they're not the raw mean. Check using descriptives to see what you think the means should be.
If he thinks interactions are negligible, should he include them an all? Also, will he have to be careful of his choice of design in order that he has replication? (I'm mainly curious for myself; I only had one lecture on fractional factorial designs.)
+1 for salesmen. Say you're interested in buying something, and a fair number of salespeople will give out their CEO's SSN.
The terminology is bad.  We don't say that maximum likelihood estimates best reproduce the population values.  The maximum likelihood estimates are obtained for population parameters by selecting the values that maximize the likelihood function. This means that the exact parametric model using the fitted parameter(s), in a way best describes the observed data. Maybe this is what he intended to say but phrased it poorly.
+1 Great answer.
In the regression context, a datapoint is a (p+1)-uple $(y_i,x_i)$. From your formulation it is not clear whether you mean uncertainty on the $x_i$'s (rhs variables), $y_i$'s or both?
I don't think there's enough information to answer any of the questions here, apart from the 'Can we compute a z-score' one (obviously you can *compute* a z-score, you just need a mean and standard deviation to standardize it with).
Can you add some more details? What is your concern here?
I do not see how you have answered any of my questions. I know that accuracy on the test set is calculated, but how do the accuracies of multiple runs end up in a single statistic?
re 95% confidence interval see, e.g., http://stats.stackexchange.com/questions/26450/why-does-a-95-ci-not-imply-a-95-chance-of-containing-the-mean. The formula is the correct one to use for normally distributed data; for other distributions there are other formulas.
That's important information that should be in the question I think.
But what if you have no way of picking a plausible null hypothesis (i.e., no two groups of people are ever exactly equal, but you also do not have enough information to predict anything better than mean1=mean2)? Explaining statistical significance without mentioning the limitations can do harm.
If you want to fit the correlation between time points, you'll need the lme function in the nlme package instead of lme4.  Also, if you want to assume that the five points are independent subsamples and everything is balanced, it's (almost nearly) equivalent to use the averages for each individual in a simple ANOVA.
Is that on the residuals?
For epidemiological stuff, I would also recommend Frank Harrell's [Hmisc](http://biostat.mc.vanderbilt.edu/wiki/Main/Hmisc) R package.
You can perform ANOVA directly as linear regression with contrast variables (such as dummies) obtained out of the two factors. Then "AB" will be a separate predictor(s) and you get their SS directly.
@whuber Grouping is something I'm assuming lonut can do if he regularly works with Excel and SQL, and there are many ways to do it. In Excel, for example, he can use a PivotTable or the COUNTIF() function; in SQL he'll probably use COUNT(*) with a GROUP BY CATEGORY clause (where CATEGORY is the computed column).
Your calculation of $p^3$ as the probability that all three drivers choose Road X _assumes_ implicitly that the choices are made _independently._ Consider that the "data" viz., that each driver chooses Road X with probability $p$ is equally consistent with the model in which (X,X,X) and (Y,Y,Y) have probability $p$ and $1-p$ respectively and all other outcomes such as (X,Y,X) have probability $0$.
I am not taking the test data into account. I hear what you're saying about the test vs train fit, it's just that difference is so wide (sometimes 4-5x better on the train data) i thought i might be doing something wrong.
http://stats.stackexchange.com/questions/15635/robust-cluster-method-for-mixed-data-in-r
Seems suspiciously narrow to me. Using functions from `library(boot)` appears to confirm this:  
> boot.ci(boot(x,function(x,i) median(x[i]), R=1000))  
Intervals :   
Level      Normal              Basic           
95%   (74.42, 78.22 )   (75.00, 78.49 )    
  
Level     Percentile            BCa            
95%   (73.51, 77.00 )   (73.00, 77.00 )
I don't think you need to remove X. Indeed, if all the predictors are significant in the equation, then certainly know I think.
@mp This isn't a changepoint problem because the times of change are exactly known.  It's really a question about regression with strongly correlated multiple measurements.
Texts? This is not very complex procedure so dig over Internet and you'll find enough. When I was writing my SPSS macro for Procrustes I used E. Lloyd and U. Lederman, Manual on Applied Statistics, Russian edition.
I don't know what the probability is of finding a given datapoint from n in a bootstrap sample, we are not given any numerical values in this question. I believe it is simply 1 trial with n datapoints, but I do not know how to find the probability that one of those n datapoints will be in the bootstrap sample.

I believe the formula for this pertains directly to the way Bootstrap sampling works, but I can't seem to figure it out from other formulas for Bootstrapping.
"the accuracy of classification is getting worse" do you mean the training or testing accuracy?
There are infinitely many possible statistics and surely many thousands have been published and used.  Please specify the ones in which you are interested.  Also, a question about terminology: to me, a "sample of data sets" means that *each* $a_i$ is itself a set of data.  Do you perhaps intend that each $a_i$ is just a number and that $A$ constitutes a batch (not a set--sets don't allow copies) of numbers?
@gung, if you look at the models closely there are 3 variables for 4 groups (2 species by 2 breeds), the 2 rabbit variables are a different coding of an interaction term.  The design matrix will not be singular (as long as all 4 cells have observations).
Your new code *multiplies* $\\binom{K}{k}\\binom{N-K}{K-k}$ by $\\binom{N}{n}$ rather than dividing by it.
By definition, $100*p$ percent of people must use A and $100*(1-p)$ percent of people must use B, unless there is a third exit, so that change in notation will simplify your graph. The same goes for number of cars thru A and B: they must add up to the total number.
You are explaining the percentile method bootstrap to the OP.  Maybe we should mention that there are the higher order accurate bootstrap CI such as iterated bootstrap, bootstrap t and BCa.  I imagine the R packages allow the use of these other bootstraps.
@nupul regarding the other ways to look at the problem, the issue with Monte-Carlo analysis is that it is prone to hide the fact that you don't understand the inputs very well under the blanket of sophisticated analysis. For this reason it is a good tool by those trying to 'hide' a lack of input data or understanding. For this reason I would recommend simple parametric analyses on key variables initially to understand how they affect the model and then report on those results rather than hiding it behind a complex Monte-Carlo analysis.
It depends on how you simulate the matrix. Your matrices do not appear to be simulated randomly, nor according to any clear rule, so it would be impossible to answer that question. What, then, is the question you really want to ask? Are you interested in creating random data with nonsingular covariance matrices?  For what purpose?
@arjsgh21 do you still need further clarification on what it means to be a closed form solution?  Because your new question seems to be about when are there closed form solutions (or not) in regression problems which is an entirely new topic and should be asked as a new question, in my opinion.
You can try to understand original BRISK paper and then ask about unclear stuff in it. 
http://www.robots.ox.ac.uk/~vgg/rg/papers/brisk.pdf - original paper.
Perfect, exactly what I was looking for - thank you!  I was stuck in the second line when working on this myself.
Your difficulty is that you have yet to formulate a research question.  What is the purpose of conducting this test?  What do you hope to learn from the survey results?  In the first image, you are merely testing whether the responses are uniformly distributed across all grades (including N/A!).  Of course they are not, but so what?  In the second image, it is not evident what the reference distribution is, but again the rejoinder is the same: so what?  What does it tell you to conclude the responses do not have a particular distributional shape?  Little or nothing, I would guess.
Most common way to check linearity is to scatter-plot residuals (studentized preferably) against the linearly predicted values. Curved or non-horizontally spead cloud on such a plot is diagnostic for non-linearity. To uncover more hidden (partial) non-linearity plot the residuals against each of the predictors.
Thanks for pointing that out Frank. I didn't know you could do that.
If you don't know how to describe it in detail, then it's not about coding but about theory. You are referencing a coding page but providing none of your own efforts. At the moment it looks like a "do my project for me" question with a bit of "make a good example", too.
Looks like @ttnphns has this covered, but since you're using R, I thought I'd also point out that a number of similarity indices (including Jaccard's) are already implemented in the `vegan` package. I think they tend to be pretty well-optimized for speed, too.
Could you explicit what you mean by "I calculate the mse for variable two and three where the third variable is my ground truth"?
What do you expect the sum of white noise terms to usually be?
Well I think I have found a solution: independent sample t-test
If P(Murray Harrison)=.055 and P(Harrison Murray) = .117, you are .117/.055=2.1 times likely to get a Harrison Murray than a Murray Harrison. If you want percentages, the chance that you have a Harrison Murray is .117/(.117+.055)=68%. This should be a start, but I am still puzzled. How did you figure out that Murray appears in FN 5,000 times and in LN 75,000 times without knowing whether it is a FN or LN??