tttjjj/elt_bench huggingface dataset:
I thought of using Bayesian methods, but I'm very new to such techniques and still trying to figure out factors and variables. But your suggestions conforms that Bayesian methods may hold the answer. I'm trying :-/
I agree with your prescription that the percentile bootstrap should not be used for untrimmed means, but I don't think it is generally true that the bootstrap-t method needs the underlying population to be normally distributed.  See my answer to http://stats.stackexchange.com/questions/39297/is-centering-needed-when-bootstrapping-the-sample-mean/49661#49661.
Do you have any idea on as to what kind of statistic to use to quantify this? I, intuitevely would like the clusters to be geographically homogeneous together i.e. maybe a lower bound on the number of adjacent pixels in the neighbourhood belonging to the same cluster could help? But that's just from the top of my head...
My problem was that in my lecture notes the professor used the notation of $\\lambda_1$ and $\\lambda_2$ to identify the parameters of the ridge and the lasso regression, and from them he derives the generalization of the elastic net, setting $\\alpha$ like in the question. Hence, I thought $\\lambda$ of R was related to these parameters! :)
Thanks for these two excellent references, I will read up on it and continue the discussion later...
A discussion of the range of percentages my sample may imply is exactly what I have in mind. Thanks again.
(+1) OK, a nice general-purpose adaptive algorithm "recently" published and already implemented in R, Python and Matlab is the [twalk](http://www.cimat.mx/~jac/twalk/). It works well but it is always a good practice to double-check using another method. BTW, MH is implemented in the R package [mcmc](http://cran.r-project.org/web/packages/mcmc/index.html). Of course, there are many others but most of them are not implemented in this level of generality and/or they are difficult to implement. Another popular area nowadays is Sequential Monte Carlo. I hope this helps.
I really don't have time to give a more thorough answer right now, I hope someone else will, or that you can find a useful discussion of contrasts and interactions online. I will just repeat that **the meaning of the LanguageL2 parameter DIFFERS** between the first and second models; you can't just say "one model says LanguageL2 is not important, the other says it is" (you should also not be interpreting p-values as denoting "importance").
1) No replacement within a draw. 2) Answer would be a probability of getting more than 135 balls that are common to all three drawings 3) The data is for significance of certain genes, but since that would add another layer of complication, I'd prefer to leave that out for now ;) Thanks for the replies!
@Jeromy Gladly.
Remember EB can fall into local minima. Try initializing hyperparameters at different locations.
@Thomas I don't think that this simple case requires convolution
Why does anyone need to consider the Jacobian?  There is no integration over the parameters.
You might find this paper by A. Gelman interesting http://www.stat.columbia.edu/~gelman/research/published/standardizing7.pdf
You are right. I did not realize that. But can't I create a function which has a curvature similar to ln(x) or (1-exp(-x)) and satisfies this property. Is there some family of functions which satisfy this property.
@Michael Yes, normally I would have edited the first answer (the present one). But each answer is long and they use substantially different approaches, with different performance characteristics, so it seemed best to post the second as a separate answer. In fact, I was tempted to delete the first after the second (adaptive) one occurred to me, but its relative speed may appeal to some people, so it would be unfair to remove it altogether.
" some people might prefer methods like support vector machines because they have the kind of cutting-edge hipster-cred which twentieth-century methods just can't match." LOL! So true. Btw you have a knack for explaining things very clearly and accurately. Thank you! I needed a 'map' on how things fit together and you provided it.
@whuber - you can add another term to the Taylor expansion underlying the delta method; if you do this, you get $\\text{Var}(X^2) \\approx \\mu_4(X)$, the fourth central moment of $X$.
@Denis, I have added in a link to the pdf. Often times if you search for the publication on Google Scholar a version is posted free to the public. The supplementary material I posted is free as well, and the R code would be insightful even without access to the paper. The description "messing about with Kdtree variants / KNN" is not specific enough to be helpful. Based on that I imagine what I quoted is not that useful unless you are explicitly interested in geographic data.
I actually scanned that at work yesterday. It's an interesting read - I wish I had more time to absorb the material in it, but I had to get what I needed and move on.
But, if we want to compare Group=B to the reference level Group=A when Condition=2 (or 3)? It is possible? And, I feel that compare if "the difference between Condition1 and Condition2 is different when Group=A vs. Group=B" is the same of compare if "the difference between Group=A and Group=B is different when Condition1 vs. Condition2". Is that correct? otherwise, what are their p-values​​?
Again, thanks. But why would you use the likelihood ratio as opposed to the wald test or the score (logrank) test. In looking at model accuracy, the coxph function in r presents (for example):
Thanks, how about " the greater change in VAR01 and VAR02 (before and after the intervention) will also result in greater changer in VAR03 and VAR04"?
Added some background info to the question
Can I suggest you add a section to your original question, marked "EDIT:....", that spells out what questions remain for you that have not been addressed by the answers and comments you've received so far.
@Yevgeny- Just to confirm, is the random effect as I have in my question indeed shrinking to the mean? Can you sketch the approach you are suggesting?
Short answer: no. Both the negative and the positive vectors are valid Principal Components.
@Glen_b, I updated the post.
@rolando2 "run 1-component PLS" means building a regression model with the first component of PLS analysis. An example of such data would be Redness as dependent and reflectance intensities over 380nm-780nm as predictors.
For R ks.test in the default "stats" package can conduct the KS test without installing additional packages.
You have made an error in copying the formula.  Your formula is *guaranteed* always to equal $0$.
ERPt of 0.0019 (0.19%) seems to be important i think? The mean of ERPt is 0.3%.
@Vokram Yes, but with a square root so that its unit is the same as the original unit of your dependent variable's. Just a nice gesture; you can do the same comparison with MS alone.
Surely it *is*, as long as the anecdotes aren't sampled with bias?
I would say that the examples I gave, [MAD](http://en.wikipedia.org/wiki/Median_absolute_deviation) and [using a t distribution as the distribution for the data](http://en.wikipedia.org/wiki/Student%27s_t-distribution#Robust_parametric_modeling) are examples of robustness with respect to data.
Add this to the end to get a graph.  It will be plain though, you will probably want to filter the lower ranking words out and only use those with a larger support.                                dd <- t(d)
library(diagram)
plotmat(dd[1:10, 1:10], box.size = 0.05, name = y[1:10], 
        lwd = 2 * dd[1:10,])
Both gated; I can't read them from this machine. But the point of my question was that I doubt the classification technique itself is doing anything but classifying. The testing is done separately, probably because underlying the technique is an assumption of equal variance. Formal hypothesis tests are generally a bad idea in such circumstances.
Tell that to the theoretical physics community...
Search for [delta method](http://en.wikipedia.org/wiki/Propagation_of_uncertainty#Non-linear_combinations)
Could you define `asymmetric loss function`?
This seems more like a comment than an answer; answers are usually lengthier and more detailed. E.g. if you included reasoning as to why Cook's distance is a good test for outliers and so one, this would be an answer.
@jbowman  I interpreted him as saying that the original dataset had "enough" females but some of the bootstrap replication datasets did not have "enough" females.  Maybe you are right, though.
@GaëlLaurans:  I agree... but I don't like to deny the premise of a question unless it is absolutely critical ... and I can definately imagine a reviewer asking for a posthoc power analysis.  However, I really liked your citations and the alternative (more appropriate) approaches you suggested.
@Tim Yes -- this is sadly quite common. About getting stuck -- there is one more issue. Here we talk nothing about the next step generation, yet obviously making big steps can be a way to escape local minima (tunneling in physical termin.). Those are of course pathologies, but sometimes people use them "because they work".
I don't understand. Did you (1) obtained two clusters of the 30 patients (some in one group, some in the other) using information from 6 variables (A–F) or (2) there are two cluster analyses, one using two variables (A,B) and the other based on patient values on four variables (C–F)? Do you want to compare (1) the two groups for their values on the 6 variables or (2) the results of different cluster analyses?
It is worth keeping in mind that non-parametric tests also make some assumptions regarding the shape of the distribution.
(1) each member of a "sequence" is a test statistic.
(2) you need to assume the marginal distribution of each test statistic.
@prasenjit: Don't forget to designate this as your answer if it turns out to be the best.
This is a terrific answer!
It would be typical to treat categorical variables as factors, all of whose levels are in or not-in at the same time. I believe functions in the R packages `grpreg`, `grplasso` and `glmnet` can all do this.
Section 10.4 in Montgomery's text discusses SPC with autocorrelated process data that might be a useful source to look at as well: http://www.amazon.com/Introduction-Statistical-Quality-Control-Editions/dp/0470169923/ref=sr_1_1?s=books
No, it seems ok to me. Recall that $\\prod_i(a_i\\times b_i \\times...)=\\prod_i(a_i)\\times \\prod_i(b_i)...$, which is exactly what you did.
No. Instead, I think that it's reasonable to say that the "inflated" odds ratio is the same as the original one, and then you'll be able to back out the "inflated" s.e. from the p-value. As an example, suppose I fit a Poisson regression to the data that is actually Negative Binomial. The point estimate of $b$ and the odds ratio will be the same in both cases, but if I stick to Poisson I will get an incorrect s.e. and the p-value. If your genomic correction does a similar thing, i.e. corrects the s.e. rather than the point estimate, then you'll be fine.
"decreasing throughout the 20th century" is too vague a phrase to allow for an hypothesis test.  One could interpret it in various ways.  1) as "each proportion must be less than the preceeding one," a condition that is clearly not met here.  2) as "the 2000 value must be less than the 1910 value," which is met.  3) as "there will be a negative coefficient for a time variable in a logistic regression (where there are 250 blue-eyed and 150 others in 1910, 1000 and 1000 in 1920, etc.)"
googling for "abductive reasoning and maximum likelihood estimation"  gives a lot of relevant hits.
Can you check my question: http://stats.stackexchange.com/questions/44464/deciding-about-dimensionality-reduction-with-pca
Also, PCA on only 3 variables is probably not useful. PCA is a data reduction method, usually used when you have many variables.
Ananth, I would also consider assigning a value to matches of dislikes.  In other words, A & B both dislike X implies some degree of similarity in tastes.  You may want to weight it differently than matching likes, but it should have relevence.
Although I know what you're referring to by "One Standard Error Rule", I strongly suspect that a lot of people won't, but would be interested in this question if they did.  Maybe you could edit to add a couple of explanatory sentences?  (Just a suggestion...)
+1for the nice answer. Except the last paragraph about normality. It is irrelevant for estimation.
gprof and valgrind are open source and widely used. The two main approaches to profiling are instrumentation and sampling. Your idea resembles sampling.
Unfortunately, you have to have at least 50 reputation in order to post comments.  The purpose of that restriction has never been clear to me, as commenting seems like a pretty low-cost way for new users to start participating.
Like I said: I agree that comparing two distributions should certainly not give less different means than identical ones (at least not consistently so). Post some code and/or example data, and maybe we can help pick up what went wrong? Note that when the means are the same, the distribution could still be very different (think of a bimodal distribution centered around the mean of a standard normal one, or a more skewed distribution).
Yes this was a nice answer. Here I gave a link to the deleted post : http://stats.stackexchange.com/questions/26556/equivalent-mixed-models-yielding-different-results-in-sas I'm going to add the link to the present post.
(+1) Great and very instructive response. Waiting for the blog post...
I think it is possible to construct an example with 6 points, where complete link clustering will give two clusters, with the largest pairwise distance among those two clusters a certain value delta. In that example, another partition into two clusters yields a smaller largest pairwise distance. It seems to me this will answer your question - is that correct? I don't have the book anymore, but this is exactly the kind of phenomenon that Jardine and Simpson study.
This makes no sense. If you apply treatments, you must expect them to affect something.
Welcome to our site!  *Please attribute your quotation* or, in general, any words that you are borrowing from someone else.
Added; Y is normally distributed, consists of approx 1000 members and 100 potential Xs (after severe correlation has been removed)
Ok, now it's clear. Many thanks for your prompt reply.
It seems to me that *blocking* denotes a more general strategy to reduce experimental errors (by accounting for the within-block homogeneity), one manifestation of this approach being the use of *repeated measurements*. I mean, blocking as seen in [RBD](http://bit.ly/AyejPq) is a way to combine randomization and control for potential confounder(s) at the level of statistical units. Blocking is also used when we cannot perform a complete replicate of a ($2^k$ or other) factorial design in a single block.
I don't think with the imbalance you have in positive versus negative instances that you should throw out information on negative cases bycutting them down to be the same as much lower total of positive instances.
Fastest and easiest way would be to hire a professional consultant.
It appears this is now properly implemented as of Scipy 0.11.0 via an optional param to specify Welch's t-test: http://docs.scipy.org/doc/scipy/reference/generated/scipy.stats.ttest_ind.html
Very true. The problem of not finding the global minimum is overrated.
@Abdel, did you find a solution to your problem?
Hello dear ttnphns - I agree it is not possible to do this in full.  However, it can be partially obtained, and I wonder if anyone has ever dealt with that...
If you have hypotheses about 10 domains, all of which may interact with one another, then you have a lot of things to test. The question of inflating and adjusting for type I error is not simple and has been discussed here before. My own view is that far too much concern is placed on this, as (at least in this type of research) you know in advance that the null isn't exactly true. ALL psychological traits are related to each other, the question is size of effect. But my view is not the only one!
I'll report my results. Many many thanks!
I already deleted it; sorry about my newbie-ness, anyway!
Check Chi's answer to this question for a start: Differences between tetrachoric and Pearson correlation and http://john-uebersax.com/stat/tetra.htm
@AdheshJosh If you *know* that it did not occur by chance, then you do not need to get any p-values - in fact, all p-values are then 0, because the p-value is only about the case where the null hypothesis is true.
Maarten, Thanks very much for your answer. It's perfectly clear and much appreciated.
+1.  Absolutely agree with stressing the real-world relevance of stats and focussing on fundamental concepts.
I have found an article with a comparison of various techniques to estimate bounds on false positive and false negative rates. Here is is http://archive.nyu.edu/bitstream/2451/27802/2/CPP-07-04.pdf.
I believe the book is not applying these theorems quite as directly as you are.  The relevant application is to $(I-H)y$ rather than to $y$ itself.  The stated results follow immediately.
The prior is Gaussian. Do you have the actual book? If not, you can read the co-author's [summary](http://www.cs.ubc.ca/~hutter/earg/papers05/rasmussen_gps_in_ml.pdf). Hanna Wallach also has an [introductory presentation](http://people.cs.umass.edu/~wallach/talks/gp_intro.pdf)
@TenaliRaman I'll try to find some references where I saw that later, and update my post. So if you think that this is a large threshold (it is in fact the the case), which thresholds can be more convenient for such a situation ?
@Néstor: I should perhaps have written that differently. I wasn't really making a statement about _statistical_ hypothesis testing, but rather making an observation about the fact that comparing model predictions with real-world data (i.e. "testing" if the hypothesis is correct) seems to be a very efficient way of doing science. At the heart of this criticism against CI's is, I believe, an unwillingness to accept this method. The kind of arguments given in the quote would apply to any statistical method - with zero prior probabilities for all null hypotheses, we'd never believe in anything.
I've read that paper.  It and a couple of others represent the entirety of the literature on the subject.
Thanks for your answer. Any packages for Python?
@Theodor; couldn't you just subtract the curve when the object is not present from the one when it is present and start from there?
@Apfelsaft, no, I don't think so.  I think atiretoo gave good advice for how to reduce the correlation, but the right answer is to change your experimental design.  If the experimental design isn't right, it is very hard to compensate for that through choice of statistical test.  In this case, a better experimental design *might* be to use a different original program for each trial, and apply one change to that program, to reduce correlation.
Thanks for the reply. But I want to know if mean of A>B and A1, A2 are the two subsets of A. Similarily, B1, B2 are two subsets of B. Is the condition A1<B1 and A2<B2 mathematically possible????? As I think it is not. I think that either both or at least one subset must have shown the same relationship in their average values as that of whole data set A and B.
...and a further comment: Similar strategies are also used for computing so-called variable importance measures, especially in random forests (but also in single trees). I also wouldn't recommend these for formal inference but for peeking into the black box they are often quite useful. See http://www.biomedcentral.com/1471-2105/8/25 and http://www.biomedcentral.com/1471-2105/9/307 and the references therein for some pointers. These are also available in "party".
I'm probably confused by the `R` notation, but doesn't your suggestion have more parameters than there are data? This confusion is not of your making: I had understood that there were $20$ subjects and just *one* result per subject (M1, M2, or tie), but the interpretation implicit in your answer is that there may be $800=20*40$ observations. Carlosdc, could you clarify this for us?
good point thanks
You do not need a book to use copulas like the one above. Once you know a function that turns two uniforms into a joint distribution with uniform marginals, you are all set! A potential reference though is [Introduction to copulas](http://www.amazon.com/gp/product/0387286594/ref=as_li_ss_tl?ie=UTF8&tag=chrprobboo-20&linkCode=as2&camp=1789&creative=390957&creativeASIN=0387286594) if you want to learn more.
Using the two extreme and middle (fifth) values of $y$, the method shown at http://stats.stackexchange.com/questions/35711/box-cox-like-transformation-for-independent-variables/35717#35717 indicates a logarithm (Box-Cox transformation with parameter 0) would be appropriate for *linearizing* the relationship. Using the first, *sixth,* and last values indicates the reciprocal (parameter -1) would be good. This suggests almost any parameter between 0 and -1 might work. The range is no surprise given how few data there are. *No* monotonic re-expression will stabilize the variation for these data.
Thanks for the helpful comments. Let me try out your suggestions.
This question appears to be too localized and off-topic because it is about a specific statistical package in a very specific context.
"Best" is really a matter of what you want to optimize.
@whuber: I see. So you need to specify the parameters beforehand?
That all sounds very reasonable. Again, I will go away and think about it. Many thanks for the consideration and advice.
Reciprocal transformation is stronger than logarithmic and often preserves meaning, as units of measurement are just inverted. For example, the reciprocal of the time to do something is a kind of speed, and vice versa. The reciprocal of miles per gallon or km per litre makes sense. Reciprocals invert the order and can be negated if that is preferable. They are naturally part of the Box-Cox scheme with that extra detail. All values should be positive for this to work well. (In principle, it would work with all values negative, but I've yet to see an example in practice.)
@d33tah Please see the discussion about this question-become-community wiki on meta: http://meta.stats.stackexchange.com/questions/2113/why-am-i-not-gaining-reputation-for-upvotes, particularly whuber's first comment to my question.
... and in example 2, the supervised analogon, PLS would actually help.
+1 Very creative and potentially quite helpful; thank you.
Thanks @Ben Bolker this covers the ground I was hoping, I might have a go at the simulation route as well. Thanks!
if you just want to control for temporal autocorrelation, you could use GEE (Generalized Estimating Equations) and do your inference with the robust standard errors.
2) This is debatable. 3) How is this more accurate, and the OP's phrasing inaccurate? 5) How is it egregious at all? Path analysis is SEM of variables for which data points are supplied, not estimated. Measurement models such as common factor models are also fit with "full SEM", which subsumes factor analysis.
If you are willing to make the assumption that $M$ and $W$ are _independent_ (normal) random variables, then you can use a fact that follows from such an assumption, namely, that  $M-W$ is a normal random variable with mean $160-120 = 40$ and standard deviation $\\sqrt{30^2+25^2} \\approx 39.05$, and proceed to determine $P(M-W>50)$ using your favorite $z$-score.
+1 for distinguishing between $\\hat{\\alpha}$ and $\\sqrt{n}(\\hat{\\alpha} - \\alpha)$ -- certainly the notation can be inconsistent.
The logistic ("sigmoid") curve is very close to straight in the region between (roughly) $-3/2$ and $3/2$. Within that region the probabilities will vary from less than $0.20$ to greater than $0.80$. Thus, when all predicted probabilities are within that range, *necessarily* the plot will look almost linear. Also, because this curve is differentiable, it is *guaranteed* to look almost straight over short enough sections, regardless of the typical predicted probabilities.  Please, then, edit your question to provide more details about the shape and range of that plot.