tttjjj/elt_bench huggingface dataset:
@efrem: To translate "a_kats"'s suggestion in R code: if you have all your "v"s in a list, you could `sapply(my_list_of_v, var)` and then call `boxplot.stats` with a suitable for your needs "coef" argument. E.g. `boxplot.stats(sapply(my_list_of_v, var), coef = 1.5)$out` returns no outliers while `boxplot.stats(sapply(my_list_of_v, var), coef = 0.9)$out` returns.
Also, thank you for your feedback. This type of analysis is new to me and the running factor analyses for my other survey questions made sense, but this particular one is confusing because of the negative statements. I should add that the factors do make sense. The items that load onto three of the five factors are coherent.
This sounds like a linear regression problem written in a slightly different-looking form. (Just out of curiosity, is $X_1$ a vector of $1$s?) If your data are not independent in the sense that $X_i$ is correlated w/ $X_{i'}$, then the sampling distributions of $\\alpha_i$ & $\\alpha_{i'}$ will be correlated. If your data are not independent in the sense that, say 2, data values on the *same variable* are correlated (eg, two observations of the same unit at different points in time), then you need to incorporate that non-independence in your model for accurate inference.
@A Giftbox. Hi. I'm curious whether my answer has helped you. If not, let me know, and I can try to improve the answer.
Nice answer. A small caveat is that the interpretation hinges on the units of $x$. If $x$ is first transformed by subtracting its mean (and maybe dividing by it standard deviation), then $a$ can be interpreted as the expected $y$ for someone who has the average $x$. Often that is more meaningful/interesting. Obviously, you can't do that and use the natural log as well. If you have a regression with no $x$, then you can interpret $a$ as the arithmetic average of $y$.
+1 thanks Nick! I need to do some more reading on the subject, however, you perfectly understood my problem. Cheers!
That is unfortunately just a byproduct of the cut-down dataset I provided, for the full list of questions (around 500 or so) there is a marked difference for certain questions in the other direction. 

I do see your point, however, but can this be remedied by picking which questions I use based on how large the difference is? I can easily pick 20-30 questions to use that have a much larger proportion of customers who passed the exam than failed. I thought that using the full dataset (although not in this short example) would be beneficial to the end result.
Thanks! Would be nice to have some time series cv code or packages in python.  Sklearn doesn't offer it.
Use summary() as it dispatches to the appropriate S3 method for class negbin. The dispersion must of course be 1, what gets estimated is theta, which is better called a shape parameter to avoid confusion. See also http://stats.stackexchange.com/questions/27773/how-does-glm-nb-work/27834#27834
That was just a hypothetical example. I'm not using this algorithm for geolocation.
**The solution for the Normal distribution cannot be correct**, because as $n$ grows large the expected squared difference between the values and their maximum must *diverge*. Counterexamples to your claim of a maximum at $1.52$ start at $n\\gt 10^7$, approximately, which is not an unheard-of sample size. Not convinced? Take two minutes to simulate it for a range of values of $n$: `sim <- function(n, mu=1/2, sigma=1/12) {
  x <- rnorm(n, mu, sqrt(sigma));
  sqrt(mean((x - max(x))^2))
};
sapply(1:7, function(k) mean(replicate(1e2, sim(10^k))))` (in `R`).
+1 That's exactly the right approach.  But given the straightforward nature of the problem, couldn't you at least sketch the nature of the solution, if not give the solution itself?
So basically I can calculate variable importance as difference between models with and without desired variable, right?
It's true because logs are monotonic.
Rscript is clearly superior to 'R CMD BATCH', which predates it.
1) Don't forget, when you add that value, you are also increasing $N$ by 1.  2) You are not adding that value to $\\sigma$, you are adding it to $\\sum Z_i^2$.
The Wikipedia entry for optimal design is quite good: http://en.m.wikipedia.org/wiki/Optimal_design It should give you enough info to narrow your question down to get a useful answer. Particularly think about what optimality criterion you want for your designs and whether you can use existing software. I also don't see what PCA and cross validation has to do with this, please explain why you used these tags because if they are relevant optimal design may not be.
I cannot see how a response that can vary between 5 and 50, with an expected value on each trial that is being ignored, will produce a high reliability for a test. Say I presented subjects with a random number of white pixels on a black screen, between 5 and 50, for 300ms each time and  you had to answer how many white pixels there were, for 200 trials, a Cronbach's alpha on those raw responses is not the correct analysis to use.
[multivariate](http://en.wikipedia.org/wiki/Multivariate_normal_distribution) is not the same as mixture. Assuming you meant univariate, then, your question is a duplicate of this [one](http://stats.stackexchange.com/a/72980/603)
By "discrete uniform random variables on (0,1)" do you mean Bernoulli random variables? In that case the joint distribution of the $U_i$ is almost trivial. Or do you have something else in mind?
Thanks! I wait a little bit more for potential new answers... but I might build on this one.
Unless you know exactly what the software is doing, it is rarely the case that creating bogus data produces any valid result. It definitely is invalid here. You need to step back and think about your assumptions. The key one appears to be that there is no variability between subjects: that all variability should be attributed to measurement error alone. Is there any basis to support that assumption? If not, you have to stop. If there is, you can test the second subject's value by erecting a *prediction interval* (search our site) based on the first subject's measurements.
In the article you referenced, I see no mention of panel data. Have you used that for panel data?If so, please be specific and provide some code to answer OP's question.
SPSS can compute Mahalanobis distances as a by-product in Linear regression and Discriminant analysis procedures. More convenient for you could be to use a special function to compute them. Take it from my web-page (Matrix - End Matrix functions). There are 2 functions for Mah. d. You'll need the second one, I guess.
What do you  mean by "check for robustness"?
I really appreciate the answer (though I'm not quite sure how to plug in my two numbers). I think I may have introduced confusion with my "wrinkle". See my update above.
The most fundamental thing to know is whether your scoring system will be *static*, which means you settle on a single formula (perhaps developed during a trial period) and apply it the same way over time, independent of how the data evolve, or *dynamic,* which means that it depends on the whole database at any time.  In a dynamic system, should the score depend on what happened during a given day or throughout a longer period, such as the entire history of the site?
Make sure you center and scale the data first!
I would suggest that you include in your question 1) The regression equation, 2) the names of the regressors and 3) What you want to measure. For example, Logistic regressions estimate the effect of the regressors on the _probability_ that the dependent variable will acquire a specific value -is this what you want to do? etc
Thanks for the readings!
Thanks! I am pleased with it too. Funny how taking logs magically made it work.
thanks, i looked at the section 12.1. But didnt see any material describing LGOCV. any particular page that you are referring to? if we are holding only one sample then isn't it same as LOOCV? i looked at the code and a bit curious. why do we have to say "index = list(TrainSet = pre2008)"? i tried changing it with "index = list(pre2008)" but got errors  :(
Yes, there is a conjugate prior for the parameters of the Gumbel distribution. Unfortunately it is not simple. You can find it in this [paper](http://www.tandfonline.com/doi/abs/10.1081/STA-100002093).
@O_Devinyak And where the distribution $\\chi^2$ comes from, is it a limit distribution?
One possibility is to consider two cases; first where $(n)=1$ (i.e. where $X_1$ is the biggest value) and then where $(n)\\neq 1$.
I don't know enough to recommend further. I was hoping someone else would jump into the conversation. :) Given how variable reduction techniques like `PCA` work, I am curious as to whether any distance metric can be used on the outputs.
@Macro The solution is $n=12$. In order for the solution to be near $1800$, the question has to be modified to read "if the required standard error is equal to $0.33$ **times** $r(x,y)$, then ..."
@jebyrnes - You can write it down or calculate it numerically, but it won't simplify much.
@suncoolsu No googling involved - I look through de Leeuw's mammoth preprint list once every 6 months to see what goodies have newly arrived.  If you can bear the fierce matrix algebra and super terse presentation, there's always something interesting to read.
It is unclear what your estimators are because you have not defined $\\hat{\\mu}$ or $\\hat{\\sigma}$. The main concern about lognormal models and small samples is that the lognormal-based estimators are sensitive to the lognormal assumption, so unless you have good evidence that this assumption is correct, it is usually better to use alternative robust estimators.
@AndyW thanks for that.
Also, is this homework?
Hrm, I'm realising now, a little late, that "modeling" is probably a bit of a loose term. I specifically meant numerical simulation/computational (physical) modeling, *not* statistical modeling. Kind of screws it up for the answerers. Sorry about that.
Thanks for the answer. But as in my original question, I'm trying to know if there are techniques that can in some manner quantitatively say something about the relative importance of attribute values of any particular data point taken into consideration.
@probabilityislogic -I agree entirely, except for when one wants the alpha value to reflect the actual long term false positive error rate (i.e. one wants to use the Neyman-Pearson approach). If you use a one-sided test and decide on the direction of change to test for on the basis of the observed effect then you get exactly twice as many false positive results as your alpha level would imply.
Normally "$n$" refers to the number of cases and "$p$" to the number of variables--the "dimension" of the data. Thus the high-dimensional case would ordinarily be understood as $p\\gg n,$ rather than $n\\gt p.$
Jaynes 1976 paper confidence intervals vs bayesian intervals.  That's at least one credible soure.  There is also Berger and Bernardo's reference priors.  Seriously, you've never heard of these?
have you checked http://stats.stackexchange.com/questions/5750/look-and-you-shall-find-a-correlation/5754#5754 ?
I once had a communication with Mike Lawrence on the ez4r mailing list in which he cited Bates & Pinheiro's recommendation to not use likelihood ratio tests for models differing only in their fixed effects: https://groups.google.com/forum/#!msg/ez4r/GQTEtNziRwE/upOoW_TLExcJ So perhaps your first idea is already bogus, but I am no expert on this.
It follows immediately from the [fact that integration is linear](http://en.wikipedia.org/wiki/Linearity_of_integration): $\\int af(x) dx = a \\int f(x) dx$. I assume you're already aware of that.
It could be applied to finance. We are using it for the optimal management of power systems though :)
It did turn out funny. I became more sure as I wrote the comment, and looked up the references :)
I think the description in your primary reference treats type B uncertainty contributions as extra variation not obtained from measurements but rather from judgement.  It sounds akin to uncertianty in parameters characterized in the prior when Bayesian me6thods are used.  NIST does not exclude type B uncertainty they just have a different definition of it and it seems to all fall under the realm of classical frequentist statistics.  Many of the things you describe in your post that apparently come from the first reference sound foreign and peculiar to me.
Can you please check your links (404 error)? You can use http://imgur.com to share images, or better copy-paste output as plain text for model results.
@longrob - the interaction terms are necessary - these are parameters I must get (= group specific slope). The only way to simplify is to remove the random effect for slope and use only (1|Taxon), in which case lmer and lme yield the same results. But, this is slightly different model.
Please say what you mean by "relevant" and "agile."  What is your research question?
Are you really sure that the logistic method fits that well? Under a logistic, you have to assume that the log odds of the probability of success is a linear function of the independent variables. That's a strong assumption. Has it been fully tested?
I voted this up as a good general answer but as a practical answer, dealing with that many data points is going to be brutal.
I've found a solution that uses another formulation, see [here](http://www2.isye.gatech.edu/~brani/isyebayes/bank/Quiz71.pdf). Glad I could help. You're welcome.
@gung That's more than a mere philosophical issue: it gets to the core reason why no statistical analysis can demonstrate the correctness of the desired conclusion. If one system is faster than the other *on just a single test* but otherwise is slower, it is still that case that people who highly value that one test would emphasize it to the complete neglect of all the others.  This is an example of a [multi-attribute valuation problem](http://stats.stackexchange.com/search?q=multi+attribute+valuation).
Mlogit is regression model. The most likely problem is that your regressors are linearly related. Using different methods will not change the problem. Supply more details about your data.
$\\mathrm{Var}(\\hat{\\beta_1}) = \\mathrm{Var}\\left(\\sum\\frac{(x_i - \\bar{x})y_i}{S_{xx}}\\right) = \\mathrm{Var}\\left(\\sum{\\frac{(x_i-\\bar{x})\\epsilon_i}{S_{xx}}}\\right) = \\mathrm{Var}\\left(\\frac{(x_1 - \\bar{x})\\epsilon_1}{S_{xx}} + \\frac{(x_2 - \\bar{x})\\epsilon_2}{S_{xx}} + \\ldots + \\frac{(x_n - \\bar{x})\\epsilon_n}{S_{xx}}\\right) = \\frac{(x_1 - \\bar{x})^2 \\sigma^2}{(S_{xx})^2} + \\frac{(x_2 - \\bar{x})^2 \\sigma^2}{(S_{xx})^2} + \\ldots + \\frac{(x_n - \\bar{x})^2 \\sigma^2}{(S_{xx})^2} = \\sigma^2 \\left[\\sum{\\frac{(x_i-\\bar{x})^2}{(S_{xx})^2}}\\right] = \\frac{\\sigma^2}{S_{xx}}$
thanks a lot , you are so helpful
@maditya: yes, that's what the fuzz parameter does.  If you have access to a good library for decision trees, random forests are actually pretty easy to implement.  Just fit ~100 decision trees, and average their predictions.  Give each one a bootstrap sample of your original dataset, as well 2 randomly selected predictor variables.  GBMs are harder.
one of these doesn't belong. the others could be googled: http://en.wikipedia.org/wiki/Stepwise_regression. Might want to expand questions to focus on specific question.
Now to your question. You surely don’t have $\\beta x^{\\beta-1} \\le 1$, take e.g $\\beta = 2$ and $x=1$... but all you need is $f(x) \\le M. g(x)$, i.e if $g(x)$ is the density of $U(0,1)$, $f(x) \\le M$.

I’ll try to type a more complete response a bit later.
OK, say it turned out 10% c diff, then what? Would you want to test that result for significance? What would a 'significant' result mean in your case?
See the Wikipedia article on [the Birthday Problem](http://en.wikipedia.org/wiki/Birthday_problem).
Anything missing in the [PROC REG](http://support.sas.com/documentation/cdl/en/statug/63033/HTML/default/viewer.htm#statug_reg_sect007.htm) documentation?
+1. The key is that this new condition *must hold no matter what value $\\theta$ has.* It can also help to find the answer explicitly for the case $n=1$.
Thanks, Henrik. Anova( ) can compute three different tests: Wald, LR, and F. I've tried all three, but it makes no difference, which I find odd. I have a feeling it the function will refuse to use tests that it decides aren't appropriate for the data...
In [at least one article](http://en.wikipedia.org/wiki/Truncated_normal_distribution), Wikipedia calls this "one-sided truncation."  Although that article focuses on truncated Normal distributions, *any* continuous distribution whose support does not include the entire real line can be expressed as a truncation of one that is supported on the real line.  The [Encyclopedia of Mathematics](http://www.encyclopediaofmath.org/index.php/Truncated_distribution) uses the terms "left truncated" and "right truncated" for distributions with semi-infinite support (and cites some authorities).
I would like to add a stochastic processes tag, but don't have the reputation, perhaps somebody could edit this for me?
Pearson's $\\rho$ does not assume normality, but is only an exhaustive measure of association if the joint distribution is multivariate normal. Given the confusion this distinction elicits, you might want to add it to your answer.
Because each of the sums in this collection is an average probability and averages never exceed the largest value, is it not obvious that when $p_x$ is the largest of all the probabilities then $A=\\{x\\}$ achieves the maximum of this set of sums? I suspect I might be misreading the question, but how?
Peter offered many useful ideas but in general, the analysis would depend on what you want to learn from the data and what you know or are willing to assume about the process generating them. There isn't one true way to analyze ratings.
Tell me if I understand you correctly: you have an interaction strength for any pair of proteins (which for example, is zero if the proteins don't interact), and then you want to have a vector of features whose value will be highly correlated with the interaction strength? Or, in other words, which will allow you to predict interaction strength between the proteins?
The titanic example at the bottom of page 6 of the manual you linked to suggests it does create dummies automatically - that formula has a factor in it. R handles factors automatically, you don't even see the dummies. [Even if `elrm` somehow didn't handle factors by itself, it's easy to generate a set of dummies in R - e.g. if you have a factor called `x`, then `model.matrix(~x)[,-1]` will generate dummies for every factor level after the first. Try `(x=as.factor(sample(LETTERS[1:3],10,replace=TRUE)))`;  `model.matrix(~x)[,-1]` and see]
@whuber Yes I would say that if you do two pairwise tests you need a multiplicity adjustment.  Now as I look at the figure and see that there are only 14 points in total, it may be difficult to formally reject equality of variances. Generally speaking variance ratios need to be large to reject equality with the F test.
@gregory_britten , the value of the features are real floating number. Maybe there are some better phrases, sorry about the confusion.
`optimize` requires two distributions to be side-by-side as I understand. In my case one is inside the other, i.e., the values from the second population are on both side of the limits.
Why don't you write down the two regression specifications, substitute for $Y$ in the first using the second, and see how it unfolds?
I am assuming you are using monthly data, so 7 seems fine. But for robustness sake, you could look at other lag lengths. Maybe 1,2,3,6,12 and 7 due to best AIC...
Hi Whuber, I agree with you that using % sugar does not take into account portion sizes, but I think perhaps it won't be too big of a problem because entree from different menus could be put on a common ground (ie I can use "one bite', for example, as the common measurement unit for % sugar in cheesecakes, salads, fried rice, noodles, etc.)
What methods do you suggest could be good for doing the comparison? Thank you!
Good point -- should have said *analogous* to the coefficient of variation (I've made the correction).
If the data are truly nominal and sparse, it seems unlikely that *any* test will have much power to distinguish them.  Permutation tests are still applicable.
I used SPSS 19. I can't add a photo or file as it seems, but the coefficients I calculate OR for are those for the predictor categories (the one I gave here is for predictor category 2 (less time spent), which is -0.474, so OR=exp(-0.474)=0.622). The coeff for category 1 (more time) is 0.111 (so OR = 1.117), and that for category 3 (no time=reference category) is coeff=0, so OR=1. This model is adjusted for gender, ethnicity, age. Outcome (food consumption) is from 1= lowest to 3= highest consumption, so that I can interpret it as moving to higher levels of consumption
@Isaac Simulations suggest the bias in your case isn't bad at all: around 0.5 on average.  It also seems the variance of this MM estimator is approximately the square of the estimate itself.  For your data this means (if the model is correct!) that you can be confident $S_0$ is between 0 and 4 or so.
I've edited your post using our [editing tools](http://stats.stackexchange.com/editing-help). Rather than Stata output, a small reproducible example would be helpful.
I have since come across this, and this seems like it could be a good way to go. However, it would be nice to let the regressors vary across choice sets. This is allowed in the varying choice set logit model, although I am not sure if there is a nice package, like the MNP package, to implement it.
@whuber: I don't think the question makes this clear. The accounts could be part of a competition (say, within a company) that doesn't cost money. If each bet does cost money, then the question of "is it worth it" depends on how much a bet costs points are "valued" (perhaps it costs $100 per bet and the points are only good for bragging rights- who knows?). I think this question needs more information.
are you trying to build your own AIC measurement?  Are you using software that doesn't include AIC for some reason?
"Pattern" is such a general concept that I'm certain you could find some kind of "patterns" even if five untrained monkeys did the classification, provided you look hard enough. If you have a more specific sense of what kinds of patterns you are looking for, that could help focus the investigation and it might even suggest good data analysis techniques.
I am not sure whether or not this will help but Cameron and Trivedi published a book on count regression models and have a second edition coming out in 2013.  Here is a link with some information: http://cameron.econ.ucdavis.edu/racd/count.html
there is a new site which might be more suitable for your question: http://quant.stackexchange.com. Now you are clearly confusing calculation with interpretation.
Is this some kind of seasonal model where the season is 10 months/years or something like that?
It seems to me that bayerj's answer requires some small modifications to fit the formula, in case somebody else needs it : `K = scipy.exp(-pairwise_dists**2 / s**2)`
A Burr distribution (at least as defined in [Wikipedia](http://en.wikipedia.org/wiki/Burr_distribution)) will not fit your data.  You need a scaled version of this distribution whose CDF $F$ has three positive parameters, $c,k,\\sigma$, and is given by $F(x;c,k,\\sigma)=1-(1+(x/\\sigma)^c)^{k+1}.$ That will fit your data nicely with $\\hat c \\approx 2.1, \\hat k \\approx 0.4,$ and $\\hat\\sigma \\approx 2460$.
@charles, yes (that's in the answer I linked in the 3rd paragraph). The way to do it is to center your variables *first*, & then form your power & product terms *after*.
Agreed that things need to be user friendly.  Since people get very protective over their working practices, any changes must make people's lives easier or they'll fail.
I think you made a slight mistake in the second paragraph: $|X|/|A| \\approx m/n$. Otherwise, what you are doing is basically reinventing Monte Carlo integration, well, the subset version I have not encountered yet, but I'd not be surprised if it's done already.