tttjjj/elt_bench huggingface dataset:
With 2 groups, all common multivariate test statistics (Wilks' $\\Lambda$, Roy's largest root, Pillai-Bartlett-trace, Hotelling-Lawley-trace, Hotelling's $T^{2}$) are equivalent: they are functions of the largest eigenvalue of $A^{-1} B$, the only eigenvalue $\\neq 0$ with 2 groups. They are strictly monotic functions of each other, and give the same $p$-value. Example: http://pastebin.com/Dq9jczE1
Thank you for your answer!  I am referring specifically to the case of a fuzzy regression discontinuity design, which in a way, is a like an IV in that it is a two-stage estimation.  I am particularly referring to the exclusion restriction in that context.  Whether that exclusion restriction is different from that of the 2SLS where there is discontinuity additionally being leveraged.
I don't know how helpful this is but the code and data are apparently available at https://github.com/mewo2/eurovision
@RyanSimmons a) to do that you'll need to first solve the math issue described in (a). b) The fact that you need to use a function doesn't make something a programming question. This is asking "which arguments do you need to give to the `glm` function to accomplish this goal," which is very much a question for a statistician rather than a programmer.
The least squares estimator may not minimize the MSE if the error/noise has non-zero mean.
Thank you very much for the example, it helped to understand the logic behind the ROC, but now I'm wondering that should I draw it base on the training or the test? Is it used to help for the cut point value selection (in this case I will draw that for the training set) or it is used to show the performance on the algorithm no matter what is the cut point (on the test set)?
@jokel Yes, this does also apply to the `lme` function.
Of course accuracy is a combination of bias and variance and an unbiased estimate with a large variance is not as good as a slightly biased estimator with a small variance.  The niave estimate of error rate is resubstitution and it has a large bias.  The bootstrap 632 and 632+ work so well because they do a good job adjusting for the bias without much increase in variance.  That is why for linear discriminant functions and quadratic discriminant functions they work much better than the leave-one-out version of cross-validation.
If you care about whether this model is any good, I think you would be extremely interested in its residuals.
I'm going for two columns for each time (one column for Latitude, other for Longitude). But I think there may be other, more elegant/eficient, solutions.
@Glen_b, I don't follow your comment. My claim is not that $\\hat y$ is equal to $X\\beta$, but rather that $\\hat y_i$ is equal to $\\bf x_i\\boldsymbol{\\hat\\beta}$. The subscripted $_i$ indexing the observations is relevant. The idea is that the predicted value, $\\hat y_i$, for a given observation is $\\bf x_i\\boldsymbol{\\hat\\beta}$. This has nothing to do w/ the population mean of $Y$. (It appears that I had forgotten to add hats to my betas, though; I've corrected that now.)
I would suggest you to work on clarity of the question; this helps in getting answers.
Thanks for your help. I've edited the question bringing more information. Do you think it would be better to use Euclidean distance?
@Adrian, Alecos Thanks for catching those typos: the $1$ should be a $-1$ and the notation for $F$ obviously is incorrect.  I'll fix them right away.
Remember AUC measures the performance *over all possible thresholds*. It would help (you as well) if you could show the curves (ideally on the same plot).
Categorizing won't make your coefficients higher or more significant, is unlikely to improve your model. The _only_ beneficial effect of it is that you may now believe the goodness-of-fit measure is computed reliably.
Your matrix approximates a uniform distribution on the unit cube $[0,1]^{100}$ in $\\mathbb{R}^{100}$. SVD computes its  moments of inertia *about the origin*. In $\\mathbb{R}^n$ the "total variance" must be $n$ times that of the unit interval, which is $1/3$. It is straightforward to compute that the moment along the main axis of the cube (emanating from the origin) equals $n/3-(n-1)/12$ and all the other moments--by virtue of symmetry--equal $1/12$. Therefore the first eigenvalue is $(n/3-(n-1)/12)/(n/3)=3/4+1/(4n)$ of the total. For $n=100$ that's $75.25$%, visible in the third plot.
An anonymous user has argued that libsvm would be able to do this. See here: [Weights for data instances](http://www.csie.ntu.edu.tw/~cjlin/libsvmtools/#weights_for_data_instances).
Yes: their difference is the variance and the variance, as a sum of squares, cannot be negative.
Good answer. Let me just add that you can use `tsboot` in the `boot` package in R to do this.
Another condidion is that your parameter cannot be on the boundary of its domain, so you might want to revise to have the first and second derivatives of $\\ell(\\theta,\\eta)|_{\\theta^*,\\eta^*}$ be the goal. Also, a simple test is to see if the log-likelihood function approaches a quadratic as n $\\rightarrow \\infty$
For scaling by 1/m, it could be for readability and interpretation i.e. a score that the user can easily understandable. However, for scaling by 1/2, it is to get rid of the 2 in the exponent when computing the derivative. For example, the derivative of x^2 is 2*x, and for (1/2)*x^2, the derivative would be x (cleaner).
Many of the tests you refer to have exactly the property you ask for: they attempt to distinguish "noise" from "signal."
Assuming Outcome is an ordinal factor, I believe lmr() will treat it as such. At least the documentation for lmr() and lmr.fit() seem to indicate that it will be. <br />Assuming your categorical variable is set up as a factor, as suggested by DWin, and your response variable is set up as an ordered factor, the inability to estimate the model may be occurring because of separation. Basically, some categories of admityear may perfectly `explain' a particular outcome.
Hmmm...ok. My real data is obviously a lot more convoluted than this, but it shares the same basic organization (in that  a very large number of points were assigned to a smaller number of voting precincts by spatial intersection.) Is there some way to estimate these boundaries and weights from data of this sort?
Do you know why your residuals are correlated? Are you making multiple measurements on the same study unit? If so, you could enter those study units as a fixed factor, or run a multilevel model & let the random effect take care of it. Another possibility is that you have the wrong functional form. You can plot your residuals against each covariate and look at them. EG if the relationship b/t a covariate & the response is curvilinear, you might have (eg) several negative residuals, then several positive residuals, then several negative ones again. This would mean you need a squared term.
Except that when you have data in e.g. $[0:1]$, you end up having an extra bin, containing $[1;1]$ only, as $1$ will not be included in the previous bins range.
Does this work for 2D matrices? Like how? If the window size is 3*3 as an example
(1) Corresponding quantiles of the observed data *DOES NOT* give a 68% interval for the mean! This was the whole point of the question I first asked in comments - don't calculate an interval for the wrong thing! (2) that's nonparametric; why specify anything about the distribution? (3) You mean 16th and 84th, in any case.
@whuber I agree that this method shouldn't be used to get data for free. But maybe for just an exploratory analysis can be useful.
I don't think @Néstor means to be harsh. In order to help you, we need to know more about your situation, your data, your goals, etc. Would you mind providing more info? At present, I don't think the Q is sufficiently answerable (despite Michael Chernick's attempt), & should be closed w/o more info. [This blog post](http://www.statisticalanalysisconsulting.com/how-to-ask-a-statistics-question/) contains useful tips about how to ask a statistical question such that it will be answerable, you may also want to read our [FAQ](http://stats.stackexchange.com/faq).
That they are sum stable and that they are the unique ones with finite variance are both forced on us by the CLT.  The interesting part of this assertion is that there exist *other* sum-stable distributions!
Cross posted here: http://www.talkstats.com/showthread.php/56616-Maximum-likelihood-estimator-in-linear-model
The 2 predictors are the lags of the error terms. Since these are not known a priori since we do not know the error terms before we begin is why this has to be treated by non-linear estimation.The confusion you are having is that a model that is finite in the past ( i.e. an AR MODEL ) is potentially infinite in the errors AND a model that is finite in the errors ( i.e. an MA MODEL) is potentially infinite in the past of Y.The reason one selects an AR MODEL versus an MA MODEL is for parsimony. Sometimes we construct an ARMA MODEL which blends both the history of Y and the history of the errors.
Actually, no. I used the multi-class variants of all 10 classifiers. Since not all classifiers give probabilities (like SVM), I treated all 10 as 'discrete' classifiers. As you may know, discrete classifiers only give a single point on the ROC. In the paper that I linked in the OP, the author recommends calculating $AUC$ for discrete classifiers by connecting this single point to $(0,0)$ and $(1,1)$ and taking the area enclosed. I did the same. For three classes I get 3 ROCs (AB, BC and AC) and hence 3 AUCs for each classifier. $AUC_total$ is simply the average of the three.
Can you clarify what you did a bit more?  What do you mean by a bunch of repetitions.  How many methods do you want to compare?  What histogram is approximately normal?
Highlight your code and click on the button that looks like binary numbers...
Have you figured this out?  I've developed metaprogramming from R to SAS for [party::ctree](https://heuristically.wordpress.com/2011/10/11/model-decision-tree-in-r-score-in-sas/) and [nnet](https://heuristically.wordpress.com/2012/10/04/nnet2sas-supports-centering-and-scaling/), and I'd like to have the same for GBM.  By the way, this is a similar question on StackExchange: [GBM Rule Generation - Coding Advice](http://stackoverflow.com/questions/9301186/gbm-rule-generation-coding-advice)
Thanks for the large amount of detail. How does the null model come into the question? It seems that only the ecological selection model matters in comparing observed vs expected.
OK thanks.  So if I understand you correctly, you cannot say that there is a 95% chance that the single confidence interval you have contains the mean?
Although it might sound like we are arguing about something, I think we generally agree. My main point is that although multiples of pie charts rarely work *well,* sometimes they appear to be among the *best* options for certain kinds of data presentation, especially if some care is taken in their construction (as we discussed with the diameter *vs* area issue).
This is way too long & rambling, w/ way too many (& unfocused) questions, to be answerable. You need to ask a focused question that can yield a self-contained answer. You are welcome to ask a series of questions as each gets answered.
From the [definition](http://en.wikipedia.org/wiki/Kullback%E2%80%93Leibler_divergence#Definition): "The K-L divergence is only defined if $P$ and $Q$ both sum to $1$ and if $Q(i)=0$ implies $P(i)=0$ for all $i$ (absolute continuity)." Therefore the answer in general is: no.
Please see the early part of the faq - the subsection relating to homework (/self-study) questions - and see the details on the [self-study](http://stats.stackexchange.com/tags/self-study/info) tag wiki to get more details.
What happens if the N(0,1) does not happen? A deterministic zero?
Yes, repeat it 999 more times. It would make better sense to align the rows, so you should end up with 17x4000 table. (You could also hit F9 1000 times, but then you won't be able to summarize the results.) You also don't seem to get the compound interest: the rate should apply to the amount in the end of the previous year, not to the amount Jane and John are investing each year. So you should have 20000 on January 1, year 1, 20000*(1+return(1)) on December 31, year 1, 20000*(1+return(1)) + 20000 on January 1, year 2, (20000*(1+return(1)) + 20000)*(1+return(2)) on December 31, year 2, etc.
thanks a lot! I think it would be great to have some simple debugging possibility wit JAGS (like browser() in R)...
It depends on how the removal occurs: are you saying each user is removed independently with probability $1/3$ or that literally every third user is systematically removed?  If the latter is the case, then consider what that does to the patterns of inter-arrival times and recall that in a Poisson process the inter-arrival times must be independent.
I'm having trouble understanding your scenario.  Do you mean $N$ samples, or one sample of size $N$?  Are you assuming a probability distribution on [0,1]?  

And I'm not sure what you're saying about the mean being in $[1/N, (N-1)/N]$.  Is this some sort of confidence interval? It doesn't seem correct though - what if $N=1$ (in which case the interval is empty)? Or $N=2$ (in which case it is the singleton $\\{1/2\\}$)?

After that, I got really lost.
@DavidMarx As pointed out to you already by cardinal, your answer gives the _marginal_ density of $\\min(Y_1,Y_2)$ and this marginal density is a valid pdf as you say, but I too do not see how your answer describes the _joint_ density of $\\min(Y_1,Y_2)$ and $V$.
Use the reshape2 package. Its `melt` function is much easier to use.
Do you have LOTS of data? Or a fairly small number of patients? And could you specify what sort of relationship you think exists between outcome and your time-series?
I really appreciate all of your help!  I thought about mcnemar's, but I have unequal sample size of male and females.  I have fewer males than females.  My understanding was that I'd have to have equal sample sizes to use matched pairs.  Would it maybe be possible to create dummy male participants (using the mean male response to preserve the correct proportion) and equalize my male and female samples so that matched pair comparison would be possible?
Do an analysis of **runs**. Also, you need to be clear about what it is you mean when you say "consistent"--I'm going to interpret it as meaning that the probability of making a shot is constant for every single shot (i.e. it is completely independent of any and all previous outcomes). Agree? Nevertheless, `do an analysis of runs`...
thanks very much, just what I was looking for.
I'm pretty sure this question has been asked and answered here.  Wikipedia gives the [formulas](http://en.wikipedia.org/wiki/Algorithms_for_calculating_variance#Covariance)
A mistake.  The names are quite similar.
@Jonah There *is* a closed formula: it is $\\frac{(-1)^m (-n)^{(m)}}{m^{n} n (m-1)!}$, where the $(m)$ refers to a *factorial power,* $x^{(m)} = x(x-1)\\cdots(x-m+1)$. For example, with $m=10,$ $n=3$ we obtain $\\frac{(-1)^{10} (-3)^{(10)}}{10^3 3 (10-1)!}$ = $220/1000$ = $0.22$.
+1 for a very thorough answer
Do you have any matlab code example for using deep learning for forecasting?
It may be worthwile to read @gung's [answer here](http://stats.stackexchange.com/a/51823/21054). Shortly: For the decision "significant vs. not-significant" or "reject null hypothesis vs. don't reject null hypothesis" it only matters whether the $p$-value is below your $\\alpha$ which you set *before* the study (Neyman & Pearson). On the other hand, you can regard the $p$-value as a continuous measure of evidence against the null hypothesis which has no "cutoff" (Fisher).
My data is i.i.d
NLP? Nonlinear Programming?  Natural language processing?  Neurolinguistic Programming?  ...
Can you describe in a little more detail the actual problem you are considering? Though somewhat tangential, there have been a couple similar questions on this site. [Here is one of them](http://stats.stackexchange.com/questions/50272/how-to-generate-a-non-integer-amount-of-consecutive-bernoulli-successes).
@loganecolss, edited answer to give some examples and point to a source.
Is there some way of implementing a 2D KS test (any variation) in R? Package?
Thanks, this is exactly what I was looking for. I think I will put off the math-theory stats course for now though.
@dksahuji thanks for INFO, also prof: Bengio is writing a DL and inital draft is available at http://www.iro.umontreal.ca/~bengioy/dlbook/
Hello Avitus. The important point is to use a "distance" that reflects as best as possible the dissimilarities of interest. The mathematical properties of a "true distance" such as triangular inequality are not primarily important. See an example in this post http://stats.stackexchange.com/questions/25764/clustering-distributions
$\\frac{1}{n-1}$ is even more "parsimonious". :-)
Maybe I'm not following - if there is an _a priori_ reason to suspect effect modification, then how do these new methods differ from, for example, including interaction terms in the list of "candidate" variables for model selection?
I believe something like `i <- interp (x, y, z)` followed by `c <- cbind (expand.grid (i$x, i$y), c(i$z))` and if you want nice names `colnames (c) <- c("x", "y", "z")` is close. I might have gotten the x and y reversed, so check that.
@amaatouq: I would have a look at [IHS](http://stats.stackexchange.com/a/1630/603) transformation (instead of the log).
@Nick, Peter: Are count data ordinal, interval or ratio?

For example count data that have Poisson distribution are discussed in categorical data analysis by agresti. Categorical data, asaik, only consist of nominal data and ordinal data.
+1. With R and Stata,`glm`s take care of the retransformation problem all on their own. Since logged outcome OLS can be recast as poisson with robust errors, which is numerically identical to glm with poisson family, log link and robust errors, this seems pertinent. Also, Stata has user-written `levpredict` to generate predictions of the levels of the dependent variable for the estimation sample in the OLS case.
@Brandon @Misha Yes, but it's just a little bit annoying, so maybe the complete answer can now be pasted here as suggested by @Srikant.
Great, thank you so much for the help!
Surely the simple confidence interval is based on the central limit theorem, not the strong law of large numbers?
The test might also be implemented in rma.uni for method = "FE". I think you still get the heterogeniety test from the fixed effects model. In any event, coming from rma.uni, the test is valid as long as the sample is large enough (small samples affect the heterogeneity tests badly). But it looks like you have heterogeneity. As a side note, you also get some other tests like the $I^2$ in that package.
Assuming that $h(u) = C$ take all integrals and see.
$E(revenue)=p\\mu_X+(1-p)\\mu_Y \\quad Var(revenue)=p\\rho_X^2+(1-p)\\rho_Y^2+2cov(X,Y)$ Do I get it right?
@Tom well, in that case, please consider accepting the answer :)
@P Schnell: In confirmatory analysis you would determine a priori what terms to keep?
"One Bayes classifier to rule them all" and in the darkness bind them?  Should we infer something about your feelings about Bayes classifiers from the choice of title?  :-)
The answer to this will certainly involve using the standard time-series tools of adjusting for seasonality and making forecasts.
Welcome to the site, Dr. Leidner. We appreciate your offering your expertise to help w/ answering questions here. One of CV's goals is to create a permanent, self-contained repository of statistical information. In that light, we prefer that answers don't need supplementary info to be complete. Would you mind editing your answer to say a little about what Krippendorff's alpha is, & why it is the appropriate tool here? (You needn't say too much.) These issues are discussed in our [FAQ](http://stats.stackexchange.com/faq), which you may want to read.
Are these percentages continuous (like the percentage of cream in milk, for example), or discrete (like binomial proportions, a count in some category out of a total count)?
By to produce $beta$'s you mean to move to standardized regression? Probably not, since your covariates are all dichotomous. And I just curious what transformation you actually used, since I posted my tips before this question.
I don't like scaling on $Y$ but rather finding changes in $Y$ per meaningful changes in $X$.  We condition on $X$ anyway so computing changes in $X$ doesn't cause estimation error in the usual sense.  I like to compute inter-quartile-range effects for each $X$, i.e., keep other $X$s constant and vary one $X$ from its first to its third quartile, computing two predicted values, then subtracting one from another.  That is the default in my R `rms` package's `summary` function.
These numbers tell you something about your data. You cannot just "increase" them without changing your data! Imagine that you recorded 100 numbers, computed the average, and then ask *"if there is a way to increase"* it? This question does not make any sense.
In SPSS, you can create user-defined contrasts through syntax. GLM is rather flexible command.
Note that the random variable is the function from $\\Omega$ to $\\mathbb{R}$, not from $\\mathcal{B}$ to $R$. The requirement is that random variable is measurable with respect to $\\mathcal{B}$.
@jp Thank you: I will look into this.  (It may take some time...)
Can you state what you mean by "recoded". EG, if it were just *reverse scored*, it would still load on the same factor, but the sign of the loading would flip. It may be just a coincidence that the only 2 recoded items are different.
10 fold cross-validation with 0.1 increment gives about 1000 runs.
(+1) since this is what I was thinking but Re: "If the data have an index ordering you can use white noise tests for time series. Essentially that means testing that the autocorrelations at all non zero lags are 0." - this logic only applies when you're dealing with a stationary time series, right? Otherwise, you could get misleading results about the lagged correlations. For example, what if only the "later" part of the time series was autocorrelated?
@whuber: One at a time and replaced each time.
The distinction between a random variable and its distribution is important. A random variable, *qua* measurable function $X:\\Omega\\to\\mathbb{R}$ on $(\\Omega, \\mathcal{F}),$ can remain the same while its distribution function can change when the *probability* on $\\mathcal{F}$ changes. Thus, it is possible for *one and the same* random variable to have different distributions.  (This point of view is essential for understanding [Girsanov's Theorem](http://en.wikipedia.org/wiki/Girsanov_theorem), for instance.)
Daniil, I've made some slight edits to correct LATEX and punctuation. Please double-check that this accurately reflects your intentions.
First of all thank you very much!!!
I shall keep your suggestion in mind while I post next time. The size of my data is so big that it would make my question a messed up crap if I pasted my data here thats why I didn't do that. 
Regards and have a nice weekend!!!
You mention a t-test in the title. Could you explain how that comes in to the body of your question?
To keep this question on topic, we will need to keep it focused on comparing the "likelihood principle" and the "repeated sampling principle": answering it in full generality requires a several-volume text.  But I suspect I'm not alone in wondering precisely what you mean by the "repeated sampling principle."  Could you please indicate what *you* understand these principles to mean, or at least provide links to the definitions you intend to use?
+1 for the humor :)
With extremely simplistic (and highly unrealistic) assumptions, $P\\{X > Y\\} = P\\{Y-X < 0\\}$, where $Y-X$ is a _normal_ random variable with mean $\\mu_Y-\\mu_X$ and standard deviation $\\sqrt{\\sigma_X^2+\\sigma_Y^2}$, can be looked up in tables of the cumulative standard normal distribution function.
+1, nb, age & gender will presumably need an interaction term as well.