tttjjj/elt_bench huggingface dataset:
I don't follow the procedure. In the simple case, when you "sort ... by their sample means" you merely identify which of $X_1$ and $X_2$ has the higher sample mean. This does not change either $\\mu_{X_1}$ nor $\\mu_{X_2}$, whence they must remain unbiased. I sounds like you're doing something additional which is not clearly described in the question.
@TooTone That was a nice guess, but it's not correct. You fell victim to the symmetry of your $x$ variable, which (when standardized) is identical to its set of negatives. *That* implies the median of $\\hat{\\beta}$ equals $\\beta$. For an illustrative counterexample do this: `x <- c(1:33 / 10^6, 1);
n <- 500;
y <- matrix(exp(rnorm(length(x) * n) - 1/2), nrow=n);
sim <- apply(y, 1, function(z) coef(lm(z ~ x))[2]);
p.hat <- sum(sim > 0) / n;
(p.hat - 1/2) / sqrt(p.hat * (1 - p.hat) / n)`. Generate Normal variates and re-run to compare.
Who said anything about countries or years?
Frank: I did not get much out reading this paper by Efron - what he does can be seen as just a sequential importance sampler which starts from the likelihood and tries to get to the posterior (which often will work). Rubin's purpose in the 1981 paper was to question the appropriateness of bootstrap but Efron's apparently reached the opposite view. David Draper revisted it this summer in his JSM course and concluded its bad except when you get to see most of the possibilities in the sample. But see here http://normaldeviate.wordpress.com/2013/06/12/happy-birthday-normal-deviate/#comment-9210
@whuber A 'same shape' assumption might be plausible, but the spreads don't seem to be: the leftmost lower peak and the rightmost higher peak are both from the same sample. That (and the related reversal in means vs pseudomedians) suggests a straight location-shift may not be the case; it looks to me likely to be more complicated than a simple removal of an O(1) operation.
(+1) I was going to suggest something similar. This is sometimes known as a 'difference in differences' estimator 
http://en.wikipedia.org/wiki/Difference_in_differences
A test isn't an estimate. Which do you want - an estimate or a test?
(+1) Copying a comment for clarification from the [MO version](http://mathoverflow.net/questions/132786): Just a small point of clarification: As I read it, $(\\alpha_1, \\alpha_2, \\alpha_3) = (1,2,3)$ qualifies under your null hypothesis, but $(1,2,2)$ and $(1,1,1)$ do not (regardless of $\\beta$). Is that what you intended? (It doesn't seem to match some of the other allusions made in the question.)
I believe there are a few other key differences.  For one, there are no wildcards (blanks) in this question.  Also, a Scrabble bag has a finite amount of tiles and picking one affects the pool of remaining tiles.  There is no such restriction here.  Is that sufficient to be considered a unique question?
I wouldn't say either of those things; I don't think either is valid. If you just want to say something about the middle of your sample distribution, you could use the *inter-quartile range*. If you want to say something about the likely precision w/ which the center of your distribution can be estimated, I would bootstrap the median.
Are you after something where the output is also discrete?
The inner product is maximized when the two vectors are collinear, and zero when they are perpendicular. The particular example I gave simply meets [the requirements of an inner product](http://en.wikipedia.org/wiki/Inner_product_space).
It's not clear what the ranking is to achieve. Why are you ranking? What will that get you?
A p-value $0,00$ is very intriguing. It means that the test statistic is huge. Note also that you are comparing distributions with different support and different skewness. What is the motivation for comparing such different models? Are you fixing the parameters *a priori*? The fit of the log-normal and the Gamma should not be extremely different. If you are fixing the parameters, this may be the reason for such extreme differences.
Forgotten bonus: unbalanced data are still useful!
The right method (a) matches your ideas of what you are estimating and (b) works well with your data to the extent allowed. (Not all samples have well-defined modes; no white magic will extract a worthwhile estimate in those cases.) That may seem a worthless answer, but you haven't given enough information to allow guidance. Why do different methods exist, any way? You might try various methods, see if they agree, relating results to various different graphs of your data and your scientific understanding. In this context, there is no such thing as "just plot[ting] the distribution"!
The question is more - What are the alternative distributions that would 'confuse' your results. Is it the one-sidedness that you seek, or what? Once you've decided on the choice you can optimise a test.
Should this be community wiki? I know that more than one reasonable answer could be given, but it seems like a question where reputation should be awarded.
But assuming that the coin is fair (i.e. given the true probability of 'Heads' is = 0.5), the probability of two coin tosses being 'Heads' is also 0.25, is it not?!?
@DilipSarwate I have change the question hope its clear. In short I need to proof each column have same probability in terms of the numbers allocation and combination with other column
@cardinal, yes.. the data obtained from FDA, it's called AERS. See the link: http://www.fda.gov/Drugs/GuidanceComplianceRegulatoryInformation/Surveillance/AdverseDrugEffects/ucm082193.htm
I realized that this answer practically duplicates @Greg's. I've added this to his answer to provide a more complete response.
Don't forget that the $J$ is based on a 10% sample, not all of $N-S$.  Presumably, there are more false negatives in the other 90% of $N-S$.
I don't know if there could be an equivalent scaling in closed form. But you could always simulate the fitted GARCH and estimate (semi)empirically.
Dear user21985 and low, please go merge and register your accounts by following the [instructions on our help page](http://stats.stackexchange.com/help/user-merge).
Sounds like a term that is specific to that author.  If you have found otherwise please share what you have found elsewhere and show why other uses of the term do not clear things up.
Hint: when $0\\lt X\\lt Y$ and $X+Y\\le 1$ for *numbers* $X$ and $Y,$ what can you say about the smallest and largest possible values of $X$? For the rest of your questions, you will find answers by consulting any Calculus textbook that covers techniques of integration.
Then large $t$-statistics being associated with small $R^2$ is entirely unremarkable. Since standard errors decrease as $1/\\sqrt{n}$, $t$-ratios will increase as $\\sqrt{n}$, while $R^2$ will tend to remain constant with increasing $n$. Why do you care what the $R^2$ is? Why do you care what the t-ratios are?
Yes, a colleague came by and I got him to help. It was also an issue putting the name in quotes in the assignment statement.
I know that I cannot sample from it directly but the point about the denominator of the above relation is that it needs to be calculated using numerical integration, and that apparently needs a Monte Carlo method integration, namely generating a sum over different values of $\\theta$ sampled from the prior!
I'm missing something: if you have to "maintain the entire vector in memory," how does this qualify as an "online" algorithm??
Have you tried an ROC plot? The AUC can be a reasonable performance metric...
Please make sure your code actually runs. There are at least two errors in your code, so it's extra difficult to figure out what is going on. Aside from that, an immediate mistake I notice is that the ``calc.v`` function you are using is for standardized mean differences, while you are apparently working with (raw) mean differences.
I think the question isn't fully specified and so it is hard to tell exactly what the OP meant without asking him directly.  Maybe instead of drafting a solution to a hypothetical problem I should have asked for clarification.  Why not do that now?  :-)
Thanks, I'll take a look at your proposal whenever possible.
@Dualinity: the intuitive answer is that this because we define efficiency as the expectation of the squared distance between the estimated value $\\bar{x}$ and the true value $\\mu$ (e.g. $E(\\bar{x}-\\mu)^2$). By aligning the loss function used in the estimation of the parameter with that used to define efficiency, we maximize the efficiency of the estimated parameter.
At first glance it sounds like a slightly more complicated version of a [*knapsack problem*](http://en.wikipedia.org/wiki/Knapsack_problem). Your cost is 'weight' in the knapsack problem (using the terminology at the link at least), and your 'expected return' is 'value' in the knapsack problem. More specifically, you have a kind of [bounded knapsack problem](http://en.wikipedia.org/wiki/List_of_knapsack_problems#Direct_generalizations), but condition 1 adds additional constraints.
I have seen that question, and your answer was good (and I do rely on a lot of the datasets that ship with R) but not many of them include defined (or should I say well characterized) answers to particular statistical questions. (although I believe that if the dataset has been used in an example to a function in the helpfile, then the package will include the results of running that example, so you could use that)
@Adam I am using an algorithm that adapts the parameters and structure of a RBFN and plugs its outputs into the http://en.wikipedia.org/wiki/Logistic_function to obtain probabilities whilst optimising the cross entropy. Looking at the function it makes sense to use weights between at least -6 and 6 to easily obtain probabilities between 0 and 1 without having to use several similar basis functions at the same place.
Hi Masterpiece and welcome to the site. Since it is a question from a class, you should use the homework tag.
i used logit regression to model default probabilities, which could be in the order of 0.01% per time period. the fact that the fail/success rates are 10% should not be a problem per se. yoor issue could be with a functional form of or the selection of independent variables. did you look at Cox hazard models?
In that case, you will have to put the marketing variable in both equations and the identification will come from the non-linearity in the IMR. Or go the Stata `tpm` route.
(+6) Nice response.
Indeed, this is taken up, to a certain degree, in the answer at the first link in the comments. :-)
(+1) I think the terminology is confusing, and I think of a model of $P(y|x)$ as being generative for the conditional distribution in contrast to SVM, say, that only models aspects of this conditional distribution for a purely discriminative purpose. As your second bullet point shows, a model of $P(y|x)$ can be augmented to become a fully generative model. The distinction is relevant from a learning point of view because we can either model $P(x,y)$ and compute $P(y|x)$ or we can model $P(y|x)$.
@Georg, as far as I know, `+0` and `-1` are fully equivalent in R. They both suppress they intercept.
@user603 - you also have the potentially massive computational advantage with the shrinkage approach.  No need to search over $2^p$ models
@horsehair, I edited the answer. Please, note that the eCDF for the scale ($\\lambda$) parameter (Figure3a, Liu et al. 2010) shows that 80% of all Weibulls had $\\lambda$ smaller than 70 (some of them had $\\lambda$ = 35). It is hard to tackle exactly what difficulties you might have into your current analysis because the subject can be very broad.
the problem with bounding the problem is that you have to know something about your data.
I was kind of worried about the sizes being too large.  Thanks for the heads up!
I do; I'm going to have to learn a lot more R, though, to share the results here, though.
Thanks very much peter and @glen-b - I don't have enough points to up-vote your answer, but I would if I could, they were both very edifying and informative.  I'm re-writing my code to add a column for coefficient of variation now, and try out the metrics that Peter suggests. I have a license to experiment with it, now that I know the rough boundaries. Thanks again - Ed
@user13443 Actually i think you can view it that you perform all the tests.  It just happens that once you reject the null hypothesis you continue to rejec6t for all the rremaining Yis.
It also applies to the subdomains. The main criterion on whether or not an algorithm will perform well is whether or not its underlying assumptions are met. No approach is truly assumption-free. Take regularization for instance, which occurs in many state-of-the-art approaches. When using regularization you assume the validity of Occam's razor.
Because in clustering, you will see this term mostly as in https://en.wikipedia.org/wiki/Longitude - from your question it is not clear *what* you want to cluster. You can cluster e.g. *intervals of time* that behave similar across "subjects", or subjects that show the same progress over time.
Thanks very much Georg. You're right about the PCA1 slope. Very useful tip also about the `drop=F` argument.
@PatrickCoulombe, yes, i think you are right. Please see the update above.
If you want your account deleted just flag the question and leave a comment to the moderator to delete your account....
This isn't correct for the edge cases. To see this, try running the following command: `table(round(runif(10000, min=0, max=2)))` It's clearly not discrete uniform.
@user721975 See additionally [this question](http://stats.stackexchange.com/questions/12386/machine-learning-cookbook-reference-card-cheatsheet) which has no definitive answer (which speaks for itself).
+1, could also look at `last_plot() + facet_grid(. ~ variable)` to isolate each scenario [depending on exactly what the OP is trying to visualize]
I don't think Claire Leslie invented the global/local distinction, but she did give a beautifully detailed description of it, with lots of references. I too strongly recommend her thesis.
Helping your field come to a consensus on just the se v. sd question would be a huge advance.  They mean completely different things.
Let's start with these considerations: what exactly does it mean for a set to be "good" or "bad"? What actions do you propose to make from your "index"? If your index is not perfect and happens to mischaracterize goodness or badness in a particular case, what would be the cost of making such an error? Is your study online--in real time--or is it retrospective, intended to evaluate the quality after the fact?
Can you give the data?
The answer is "yes."  If you need more than that, please amplify your question.
I understand you forced the means to equal $0$ and the variances $1$ for simplicity. Let $u\\triangleq y-\\rho x$. Then $du = dy$, and by defining $\\mu_U\\triangleq 0$ and $\\sigma_U\\triangleq\\sqrt{1-\\rho^2}$, it's clear that the integral has to be $1$. That is, $\\rho$ disappears indeed!
@Mark--the output of a beta binomial is a count distribution, just like with the binomial or Poisson. The difference is that the count distribution has mire weight in the tails--the probabilities of large IRS all values is greater. The part about the logit has to do with the guts of how to fit a glm--it would show up if you were using a binomial distribution as well.
What is the subsequent analysis?
Some pointers, but someone has probably some more thoughts to share: [Uniform prior in mcmcsamp](http://bit.ly/znd2BP); [lme4::mcmcsamp + coda::HPDinterval](http://bit.ly/xxyCG4).
Based on my fleeting understanding of dependent errors, not using an AR term is going to throw off the confidence intervals for my estimations, isn't it? Let's just say I want to build a bunch of 95% confidence intervals for each parameter and say "are they different?" (allowing for Bonferonni correction). Aren't the s.e.'s of k (as calculated by R or Matlab) way too conservative if my errors have dependency?
So essentially you're trying to approximate an integral of the form $\\int g(x) f(x) dx$ where $g(x)$ is the solution to your optimization problem and is expensive to calculate. Have you consider some quadrature routines? You might be able to estimate the integral with far fewer evaluations of $g(x)$ than the monte carlo method you want to use. If $f(x)$ is approximately the Gaussian density, you could estimate the integral very cheaply using Gauss-Hermite quadrature.
I think this will become apparent if you set it to 1, then set it too 100, and compare the results.
@Srikant, you are right, to estimate the parameters the factorial is not an issue, but in general you will want the value of the likelihood for a given model, and you would have to use the factorial for that. Also, for hypothesis testing (e.g. likelihood ratio test) you will need the value of the likelihood.
I didn't expect I'd have to quote chapter-and-verse to a high-karma user, but... [SE guidelines disagree](http://meta.stackoverflow.com/tags/link-only-answers/info). So does the "[How to Answer](http://stats.stackexchange.com/questions/how-to-answer)" page on CV. But thanks for the reference, I'll make sure to look into it.
And no--no need to compute the Jacobian here (unless you want to for some reason).  The likelihood here is just the product of the likelihoods from each observation. With a little effort you can collect the terms in the product of likelihoods and rewrite them in terms of the sufficient statistics. (Note, as JMS says, you DO need to know $N$).
It makes so much sense when u mentioned that negative costs indicates that linear regression might not be appropriate. I will continue my analysis and add some interactions. Thank you.
I have tried it for experiments. It has good classification accuracy. The author has taken the number of features close to half of the original size of feature set. I have tried this for other data sets too. The authors have also given plots of entropy vs reduced feature set size for 3 datasets. Th entropy drops sharply at half the original feature set size for these 3 data sets used in the paper.
Read this recent question http://stats.stackexchange.com/q/79968/3277 with further link in it. Since PCA does _not_ know about the existence of the classes it does not _guarantee_ that any of the PCs will be really good discriminators; all the more that PC1 will be a good discriminator. See also two pictures as example [here](http://stats.stackexchange.com/q/12861/3277).
I'd encourage anyone using Mplus to just ask the Muthéns themselves on the Mplus forums. Their customer service is amazing! Do let us know the resolution back here if you follow up on this particular question over there though!
+1 for the simulations and re-usable code. Nice!
Very good point: predictability is not the only game. I was answering the original question, not claiming to provide a knock-down argument for all pathological cases. Arguments can collapse, or be refuted, for different reasons. For (constant, constant') no correlation is definable geometrically; For (constant, y) or (x, constant) no correlation can be defined consistently. I usually try to teach this as saying that you can't define a correlation if there isn't a linear relationship, but then the rest of the argument consists of a Pickwickian interpretation of "relationship".
@Henry made some excellent points in his answer. We're both warning you that you probably don't have the data to be able to test your hypothesis. In addition, because you don't have control over any variables, you're going to be looking at correlations and not be able to deduct causality.
Cross validation still needs the user to take care that the splits are done independently. My guess is that this is a problem in geostatistics (even more than for time series). If that is not the case, then the resulting optimistic bias will also influence the observed variance.
It is also a compound Poisson distribution where you sum a Poisson-distributed number of logarithmic random variables.
@whuber I *don't* think it's a duplicate, since my answer there didn't take account of the heavy discreteness, not mentioned in the original post. I suggested the new post rather than have to extensively modify analysis that didn't necessarily applied to the additional context (it might work of the OP's $n$, but what would be a reasonable $n$ with data like this?). I think this new question deserves an answer that deals with the heavy discreteness (very high proportion of zeroes, with a few larger values), to investigate the sort of sample size it's reasonable to do a t-test for in that case.
"*is it Gaussian noise or random error*" -- yes, that random error term is usually taken to be Gaussian noise, though if you're only estimating the coefficients (rather than computing intervals or doing hypothesis tests) it doesn't have to be Gaussian. If it's Gaussian then LS is optimal in several different senses at once. If it's not Gaussian then you still have that it's best linear unbiased.
Could you add a bit more detail on your question.  What I would like to see is an explicit statement of the parameters: such as $(x_i|\\alpha x_m)\\sim Pareto(\\alpha,x_m)$ for $i=1,\\dots,n$.  Then define $x_{min}$ - is it a statistic (i.e. function of the $x_i$) or a parameter?  Similarly, is $x_{min}^{'}$ a statistic or a parameter?  what about $\\alpha^{'}$?
@tristan hi, just consider it is a simple number matrix. The first column is label. The label has simple linear relationship with other col. The label can be 1 or 2 or 3. For example, label of test data is 1,2,3,1, and prediction is 1,2,2,1. So just tell me how to calculate the true negative?
I don't think there exist "the best approach". There are many ways to analyse or report peaks in time series. Your question gives no clue to understand what you might be after. Maybe you consult with articles in your field, for hints or benchmarks.
Ah thanks! A very nice and complete answer! This I can use if I do a linearised fit, which I also sometimes do. I hope you dont mind that I accept  Harveys answer, as in this case my question was not about the linearised fit. Still a useful answer though!
(+1) Thanks for nicely expanding your comment into an answer.
+1. Also, I hope you will return at some point to do/show the simulations/results, for everybody to see that you were correct.
ADDITIONAL INFO FOR CASE 1)
 x=[0.94736842105263153]; alphas = [0.032034681077891182]======> implementation 1: 1.05372888806 ; implementation 2: 0.   This happens very often when I have only 1 value and 1 alpha. Does it make sense? Also, why do the two densities differ?
They certainly are, yet AFAIK neither can calculate the area under precision-recall curve.
Your counterexample presumes $X$ has no probability of being zero.  Simpler counterexamples are easy to come by: *e.g.*, take $X=Y=W$.
@DilipSarwate That would be (b). I want a way to check whether the joint distribution is in fact normal.
Since I am getting several downvotes,would someone explain the justification for it?
I left this *post notice* because it's usually better to provide a summary of the main points you want to emphasize rather than link to a blog without further explanations. Moreover, the purpose of this site is to build a collection of knowledgeable responses to specific questions with minimum dependencies on dangling or ephemeral links. So unless you can guarantee that the above link will be still alive in 10 years, say, I would strongly encourage you to summarize its main points in the present reply.
@evan54 I rewrote my answer to contain my interpretation of the setting problem 3, more explanation about why the situation is not symmetric, examples etc.
(+1) Particularly, for the counterargument set up.
@mpiktas: You've assumed the questioner is trying to estimate an underlying *model*, when all he said is that he's fitting a couple of linear regressions. http://www.jstor.org/stable/2676681
Are any of the predictors significant in the logistic regression?
+1 for "you have a continuous predictor you can treat it as such and examine potential nonlinear relationships using polynomial terms, fractional polynomials or restricted cubic splines", which I think is the real solution here.