tttjjj/elt_bench huggingface dataset:
The issue with confidence intervals in repeated experiments, is that in order for them to work, the conditions of the repeatable experiment need to stay the same (and who would believe that?), whereas the Bayesian interval (if used properly) conditions on the data observed, and thus provides allowances for changes which occur in the real world (via data).  I think it is the *conditioning rules* of Bayesian statistics which make it so hard to outperform (I think it is impossible: only equivalence can be achieved), and the automatic machinery which it achieves this that make it seem so slick.
Does the opening line of the link that you posted not answer your question?
+1 Very interesting example! It's also very nice to see someone mentioning Miguel Hernan. As a side note: His [causal inference book](http://www.hsph.harvard.edu/miguel-hernan/causal-inference-book/) is (still) freely available.
@Jan-Henk, I have added the actual computation of the probability for a mixed strategy. Now it is crystal clear that the mixed strategy falls in-between the two pure strategies.
@Shane, I guess so. I hadn't seen the other two conference questions before I answered.
@AndrewCassidy Yea, that's it!
Can you give me some pointers to any existing implementation of adaboost or a paper that does "feature ranking" and training in the manner that you've mentioned?
Are the failures a subset of those patented?
The median. That is explicit in the document you cite. By default, the median of each set of values is shown as a marker (point symbol) and the mean as a line. (Not your question, but the representation here seems rather busy to me and transposing to horizontal would avoid the clumsy display of labels such as "Strong Democrat".)
Thanks for the response. I looked at OpenMx but it seems to have a rather steep learning curve. I'll try to replicate the example with OpenMx when I get the time. Until then I think I will stick with depmixS4.
You're correct that it indicates right skewness. I'll try to locate some of the posts on interpreting QQ plots.
One small thing, rather than constantly resizing your vector, start off with: `x = numeric(100000)` This will be much faster.
Imagine k1 = k2. Then, h = 0 everywhere. Its integral cannot be 1, which is a requirement of kernel functions.
How about this? http://www.quora.com/What-s-the-best-parallelized-sparse-SVD-code-publicly-available
No, I don't think you do. But it may be tricky to handle a quadratic relations for just ONE person (just as it would for an OLS model).
I think that this is a dangerous answer. If you do not shuffle your data set and do stochastic gradient descent, you might get weird oscillating behaviour where you run in circles. Basically, you might follow a fixed trajectory through function space and follow it again and again. Actually, there is absolutely no reason to NOT shuffle your data set if it is finite. The only reason not to is if it has no effect, which is the case in batch learning. Otherwise, *always* shuffle since it does no harm at all. Another exception to this might be if you try to learn a non stationary function online.
Ah, I see. Thanks for the heads up. :) Well to be honest I haven't really tried too many different things. I'm very perplexed as to how to approach estimating "a part" of the parameter of the distribution. Which I guess is why I'm on here in the first place. :p
By the way, if you feel my answer adequately addresses your question, feel free to accept it (the same applies to http://stats.stackexchange.com/q/71404/1934).
I posted this question as a means to get some help from the general public. I did not intend this to be a criticism on anyones work. I work in the same industry and sometimes the profitable and meaningful ideas come simply from things that don't make sense in academia. 

Corey has to all intent and purposes offered full transparency and I really appreciate that.
What, specifically, were you doing with your resampled sample?  For example, were you calculating a sample mean, then looking at the distribution of that across all the resamples?
But couldn't you simply include dummies for each set, and dummy$\\times$variable interactions and thereby fit different hyperplanes to each set within a single model?
Your question asks for the asymptotic distribution of a sum of iid variables, as multiplied by a constant ($2\\left(1/\\beta_0-1/\\hat{\\beta}\\right)$) to which a constant has been added ($2 n\\alpha \\log(\\beta_0/\\hat{\\beta})$). Surely you know all about [that kind of asymptotic distribution](http://en.wikipedia.org/wiki/Central_limit_theorem#Classical_CLT)!
@gung MathOverflow is an SE 1.0 site, so it's not technically affiliated with the Exchange any longer.  So, there's no migration path, but I think both sides probably feel the same way about cross posting.
Gelman and Hill do not really use the marginal effects, but rather graphs, that are based on predicted probabilities. Have a look at chapter five, starting at page 79.
I can give you a reference. Look up the Palm-Khintchine theorem on Wikipedia.
Because you standardize the columns you are decomposing a bivariate correlation matrix. Whenever its off-diagonal element (the correlation coef.) is non-zero you always get eigenvector matrix with elements +/-.70711 which means 45 degree rotation. But when the correlation is zero there is no need to rotate at all, and so you get eigenvector matrix equal to the correlation matrix.
"arrived at" just means "decided to use".  "consolidated dependent variable N" means they constructed the dependent variable (e.g. perhaps from several raters, each rating 1-5 -- impossible to say from what you wrote so far). "linear multiple regression" is a standard statistical technique; I'm not good enough to explain this in the characters I have left.
Hey Chl. I will try this mclust package and see what it can do.

Thanks for the hint :)
Are you asking for the proof in order to complete a self-study exercise, coursework, or, is it for exam preparation?
nice try. Yes, I have checked the package, but claiming that it contains all data, functions and most scripts is an exaggeration. It is not complete, and it does not contain the data set in question.
$X$ is not a statistic, $a$ and $b$ are statistics - these would be functions of the $p_i$ and other sample properties.  $X$ would be a parameter which is not fully known from the sample data at hand (such as a population proportion).
@toomuchpj The [poLCA](http://dlinzer.github.io/poLCA/) R package offers some facilities for Latent Class Analysis as well.
Outcome variable is not 0 or 1, but it is efficiency %. I am not interested in testing whether they picked the M&M or not, but in efficiency of picking M&M's based on spoon design. And does left or right hand plays any role there.
[Check](http://forums.eviews.com/viewtopic.php?f=7&t=465)
That, at least, is possible to make statements about. :) It would help if you told us which estimators you used for the parameters.
The primary assumption in textbook k-means is that variances between clusters are equal.  Because it assumes this in the derivation, the algorithm that optimizes (or expectation maximizes) the fit will set equal variance across clusters.
@MattKrause I did not mean to suggest that one should not use complex models. They just need more data in order to generalize well. So usually if you do not have a good reason to use a complex model, it would be better to keep the model as simple as possible (which is essentially Occam's razor).
As a matter of observation alone we can see that the distances between the centroids of the sampling distributions get reduced as the centroids of sampling distributions moves away from .5 (probably related to point f above).  This effect tends to counteract the tendency for the high frequencies of observations for the more common nearly equal successes in group A and group B case.  However, giving a mathematical solution as to why that is or why it should yield normal distributions for certain bin sizes isn't anywhere near my territory.
There are two things that don't work with update (N.B.: update.lm is deprecated). First, it uses formulas. This in itself makes execution 400% slower than lm.fit() Second, it *reestimates* the entire model. There are no efficiency gains here.
(+1) I liked both answers, this one probably a bit more.
Also (as an add-in to David's answer): read this (http://stats.stackexchange.com/a/2498/603) and set your sample size to 100 and see the difference it makes.
Sounds good to me. For your second approach (permutation test), a standard reference seems to be [Good, PI (2005) Permutation, Parametric, and Bootstrap Tests of Hypotheses](http://www.springer.com/statistics/statistical+theory+and+methods/book/978-0-387-20279-2). Is there a specific reason for choosing the ratio of your performance measures as a test statistic instead of their difference? WRT boostrap CIs: BCa intervals are supposed to be more accurate when the bootstrap-distribution exhibits skew and bias.
Is there another way I can do anova besides the ez package?
Thank you.. this is exactly what I was looking for.
@Nick +1 while no justification is offered that the two population sds should be the same, I believe that's the required assumption, as the request for $Z_\\text{crit}$ and $Z_\\text{obt}$ information makes clear. This assumption is no more odd than the (very) common use of equal-variance two-sample t-tests which make the same assumption with no more justification offered than is given here. However, you did remind me of something I intended to put in my answer - the need to make that rather significant assumption explicit.
Ok did not know that.
@user35573 while you certainly 'declared p to be finite' it was not clear in the original what, exactly, you were saying was finite - when you used the word 'density' I assumed you meant that the density was bounded. Presumably you intend that the sample space of $X$ is finite, not $p$ itself - which must be finite, since it's now clearly a probability (always between 0 and 1)
But the question is: how would you verify that to someone on the other side of the world?  See item b) in the question.
Welcome to the site, @Pawan. There have been many questions here asking for recommendations for books, you can search the [tag:books] tag for some different versions of this common question. To make your question distinct, I edited the text to focus it more explicitly on books specific to *making business decisions*; make sure the question is still acceptable to you.
If you want a pretty comprehensive treatment of this check out this paper from NIPS this year: http://papers.nips.cc/paper/4928-understanding-variable-importances-in-forests-of-randomized-trees.pdf
your gaussian variables are iid ?
We interpret them as you write them, Val. Consider for instance your first comment: "I am more interested to know why is the probability of having success in n trials is not 1?" Call me naive if you like, but I read this as referring to "$n$ trials" and I cannot find any unambiguous reference to "more than $n$" of them. Successful communication requires both parties--author and reader--to share an understanding. When you find that readers have different interpretations of what you wrote, you should not be so quick to call them "naive," but should instead consider how to clarify your words.
It's unclear what a "combination" is.  (You do not use the word in its [usual mathematical or statistical sense](http://en.wikipedia.org/wiki/Combination).)  For instance, would AC-BD qualify?  B-AC? BC-A? If you can't define this, then it would help to see several complete, correct examples.  Also, for this question to have some relevance to the site, please explain its connections with statistics or machine learning.
Is this a question about *describing* the data or about making *inferences* about their parent distribution?
I believe I've asked a genuinely new question, which is for feedback on my regressions and my interpretation of their coefficients. If you concur, I'd appreciate your un-marking this question as a duplicate. Thank you!
Nice example, whuber. I agree that one fixed threshold makes little sense for a large field. On the other hand, the health sector is a general field too, and there is the overview document I cited that lists several reliability standards depending on specific domains. So maybe there is something similar for computer science or randomized algorithms?
Right.  Rewriting the x3=1 case as E[y] = B0' + B1'x1 + B2'x2 shows that we're really fitting two models (with a common error variance): one for non-smokers with parameters B0, B1, B2 and another for smokers with parameters B0', B1', B2'.  "Parallelism" means B1 = B1' and B2 = B2' (which says, geometrically, that plots of the fits are parallel planes).  In terms of the original variables these hypotheses are B4 = 0 and B5 = 0, exactly as Srikant has proposed.
`prop.test()` is appropriate only if the proportions are derived from counts.  Sometimes data arises as a set of proportions with unknown denominators.
Allocating degrees of freedom to the subject effect also decreases the model's power to detect differences in the condition effect.
GEE is a method of fitting models as I described.  We usually don't use the phrase "GEE estimator" for describing the variance-covariance matrix estimator but focus on the name of a particular estimator that can be used with or without the context of GEE: the sandwich estimator.  The cluster bootstrap is a competitor of that.
As a general policy, when you've forgotten (or don't know) terminology, I find it best to explain your question in terms that are as simple as possible. 

With many of my clients, my first goal is to get them to explain their question in simple, substantive terms. Then I can (often) figure out the statistics.
If you have 5 candidate variables, they are all binary, and you don't posit any interactions, you would need at least 75 events and non-events total.  But since you need 96 observations to estimate the intercept reliably, somehow add that into the rule of thumb.
Now I just need to figure out how JMP works with long and wide structures…
@Richard The pgf approach works just fine--in fact, I used that method (in *Mathematica*, where it's a one-liner) to verify my example here. But under the hood, the product is carried out as a convolution, so it amounts to the same solution.  BTW, there's no need to take a derivative: you only need to expand the product as a polynomial in $s$ and pick out the probabilities from the coefficients.
These ys are measurements at different locations
Thank you. There is one think I *still* don't understand: where do we put information about $\\theta_0$ in the bootstrap? Where $H_0$ is false, the $\\theta_0$ might be considerably off from the true distribution.
Hi, $ n $ is the sample size - and is equal to $ n=x+y+z $. The chi-square tests $ D $ and $ X^2$ are multivariate tests, so you could create a 2-d region for $(x, y) $ such that $ X^2\\leq 5.99$ (similarly for $ D $)
I would like to second this. If you're looking for a fast and easy way to interface compiled code with R, `Rcpp` with `RcppArmadillo` is the way to go. Edit: Using Rcpp, you also have access to all the RNGs implmented in the C-code underlying R.
@Stuart To measure (device independent) cone excitations corresponding to a scene in nature, you need [multispectral imaging](http://en.wikipedia.org/wiki/Hyperspectral_imaging) and apply the [cone fundamentals](http://www.cvrl.org/cones.htm) (their spectral sensitivity curves) at each pixel. From raw cone excitations, you can convert into any perceptual color space you want, including the ones used for calculating distance measures.
Maybe a better place to ask this question is here: http://academia.stackexchange.com/
I would be wary of *anyone* claiming some deep insight that would revolutionize "science".
Please translate this sentence to English: "Nadat ik die tweets had verzameld ben ik de bijbehorende quotes (per uur) op gaan vragen en heb daar returns van gemaakt om te zorgen dat de data stationair is."
After reading the comment from whuber, I think this question would be much clearer with a definition of winnings.
It's not obvious to me that this would be possible. The model could be completely different without an intercept, and I'm not sure why there should be an easy path from one model to the other.  Can you explain why you think this is feasible?  Perhaps you could use another method instead?  For instance, `bayesglm` in the `arm` package can fit regularized glms without intercepts (although they aren't sparse).  Might be worth looking into other options as well, rather than forcing glmnet to do something it doesn't want to.
I don't think we're communicating: your question states that $z=g(x,y)$ and it gives the formula for $g$. That is the formula I used to create all three plots. The 3D figure perhaps most clearly shows what is going on: you use one formula for $z$ inside the unit circle and another formula outside the unit circle. This is what creates the visible breaks in all these plots.
Can you put the link of the lecture you mentioned as reference?
Covariation among the variables is the covariance of the columns of the X matrix in the regression, i.e., your $iv1 \\dots iv6$ variables in your data sample.  The covariance of the coefficient estimates is the covariance between $\\hat{\\beta}_0 \\dots \\hat{\\beta}_6$ ($\\hat{\\beta}_0$ being the estimate of the constant term.)  The vcov function applied to the result of your regression (vcov(lm(dv+iv1+iv2+iv3+iv4+iv5+iv6)), for example) will give you the covariance matrix of the parameter estimates.
Thank you Peter! I have no idea how I missed that. I really appreciate it.
Are you interested in marginal effects or predictions? In the latent or the actual variable?
+1 This is an excellent answer. I appreciate your suggestion about using bootstrapping to test for median significance. Do you know if this can be done out-of-the-box in R? I mean, are there any standard functions to do this? To answer your question, yes, I am interested in the median of X being higher than that of Y. As per your last comment: This data is the amount of water flowing on a pipe taken on an hourly basis. Considering that stationarity is not seen in real-world processes, do you have any suggestions on how to handle this?
could you please add the self-study tag?
Note that use of SPSS by poster is immaterial to the question. I've removed the tag.
You could do a nested anova to identify whether any of the treatment effects are significant. A good reference is _Biometry_ by Sokal and Rholf.
Because the Taylor series for log has a relatively small radius of convergence, caution is advised in applying these approximations.
Thank you for the answer! I'm not sure I fully appreciate the answer of the first part, but do understand the general problem (this is more my lack of understanding of the topic). As for the second part, I do understand this and it's definitely useful! I had originally tried this and thought that I wasn't doing the analysis correctly(!). Your comment of "other measures of effect size" is also very useful - of course the data I'm looking at is being deemed non-normal; at least by the gvlma package/function.
Are you asking about how trustworthy the automated procedure is?  if so there isn't enough information i think.  What you need is the number of correctly identified errors, the number of incorrectly identified errors, the number of correctly identified non-errors and the number of incorrectly idenified non-errors.  I use the word "correct" in the sense of agreement with manual review.
I just can't understand why people think this swapping one is 'simpler' or 'more naive' than FY... When I was solving this problem for a first time I have just implemented FY (not knowing it has even a name), just because it seemed the simplest way to do it for me.
Are you talking about a sample of $x$'s to which you're trying to find a local minimum of a density, or is this a set of observations of (possibly noisy) pairs, $(x,f(x))$? A mix of gaussians would be able to locate a local minimum of the fit in either case, but the exact estimate of that location may be somewhat sensitive to the gaussian assumption. An alternative would be to smooth the data using kernels (KDEs in the second of the two cases I mentioned). Another possibility is to use log-splines (log-spline density estimation in the second case).
Change scores in general.  In many settings change cannot be interpreted without reference to the starting value, so the notion of analyzing change is problematic (as opposed to predicted the final value conditioning on the initial value).  Details are at http://biostat.mc.vanderbilt.edu/ManuscriptChecklist under "Inappropriate choice of measure of change".  Look at it this way: It is statistically invalid to analyze change without conditioning on the initial value anyway (b/c of bias or power loss) and it is complex to have the initial value in 2 places in the model.
I edited the question. $Y_n$ is the random variable that models the whole experiment whereas the X's are the binomial random variables of each step. Since I don't know how to call the distribution of $Y_n$ I said "unknown" (potentially "sum of product of binomials").
This fact, and more, is a theorem of Renyi. Note that you do need some caveat regarding continuity of the distribution, but you don't really need to do the reduction to the uniform distribution that you've done. It follows from symmetry considerations. :-)
@MichaelMayer I added a (simulated) data example to the post.
@ashaw: Comparison of conditions experienced by the same individuals is a standard methodological approach that increases power by permitting removal of between-Ss variance that usually obscures effect variance in completely between-Ss designs. Regarding the computation of the difference score CI, I have edited the post above to clarify that on each iteration the condition means were computed and collapsed to a difference, yielding a distribution of differences from which the 95% interval.
"$c_{t-1}$ is a typo and has now disappeared...
@chl: nice catch. I'm putting that paper on my queue ;)
Why not have "Number of Black parents" as a single IV, with three levels (0, 1, 2)?
Related: [C puzzle: Make a fair coin from a biased coin](http://stackoverflow.com/q/5429045/1378672) and [Getting a Fair Toss From a Biased Coin](http://www.billthelizard.com/2009/09/getting-fair-toss-from-biased-coin.html).
@StumpyJoePete I edited my post. Thanks.
"*because even though Y(n) would maximize the original pdf, there could be y's that fall outside of the range [0, 1/Y(n)], thus making the likelihood function equal zero. Instead, we choose an MLE that will allow us to "accept" the most y's*" -- Smootie, you don't seem to correctly characterize how it works there. You simply maximize the likelihood function. There's nothing fancy going on. You are leading yourself into misunderstanding. I suggest that you go back to absolute basics - I'd start by attempting to respond to the question I asked you under my answer.
First define how you are classifying predictions. Then ask yourself if your classification scheme (which is not an inherent part of logistic regression) is reasonable. My goal was merely to point out that the "correct guesses" view of performance is flawed in certain ways.
@user2907250 i used the formula of eigenvalues and eigenvectors: A*x=lambda*x   http://en.wikipedia.org/wiki/Eigenvalues_and_eigenvectors
@JakubHampl Not really. You say "We also have the proportion of overlap with the blue box which are the numbers in the table below", but there are a lot of numbers there, and an aggregate row with odd min/max. Is the proportion of which you speak the first three rows of the `avg` column?
IMHO that really matters is _why do you need either of them_? And your factors are constructed not to be orthogonal to each other combinations? Are you trying to construct just a some sort of index for separation purposes or do you need to implement rotation of the factor space to have a nice interpretation?