tttjjj/elt_bench huggingface dataset:
I don't know my survival data, but google seems to: [Wilcox Test](http://www.r-tutor.com/elementary-statistics/non-parametric-methods/wilcoxon-signed-rank-test) And reading the docs for `survdiff` setting `rho=1` makes it a Peto test...
Multivariate or univariate ?(it makes a huge difference in this context)
No prior information used hey?  How did you specify the functional form for the likelihood function then? :-D
then just multiply the proportions by the total sample size to transform into a table of counts, and apply the chi-sq. method corresponding to your first example.
(+1) Nice. Very efficient way to compute the table of transitions.
in-sample or out-of-sample? _i.e._ are regression trees better at _remembering data_ or _predicting responses_?
Interesting question; welcome to Cross Validated!
If any of this is at all useful and you'd be interested in seeing a further development of the simple adaptation of the "independent scoring" model to your scenario, let me know and I will try to write it up (hopefully a little more concisely) as a separate answer. Cheers.
@whuber I see. So you have argued the pros and cons of both approaches.  Assuming the lognormal is a good fit to the data which approach would you recommend to the OP?
Here it is kind of the opposite.  The interaction (of interest) is only significant when the main effects are not in the model.
+1 The difference would be important, perhaps, for [Buzz Lightyear](http://en.wikipedia.org/wiki/Buzz_Lightyear#Cultural_impact), who could get to infinity--or at least knew he was headed there.
Thanks, would have a look at the paper.
This project is not a simple one. In such cases, I think it would be better to just use some professional operations management software (any number pop up on a google search) and figure out what it is they are doing. Very smart people have spent a lot of time thinking about each of these questions (There is a lot of money in this question..Walmart,Target etc. have entire teams working on this problem). I think its best to utilize these advances than starting out on writing code yourself. Though this may not be the answer you want, this is my 2 cents
The options you offer are not the only options. For example, there are [overlapping samples tests](https://en.wikipedia.org/wiki/Student%27s_t-test#Overlapping_samples).
By "the same" you mean in the same approximate range with a similar distribution?
The answer depends on what you are actually summing.  For instance, obtaining an answer is easy and instantaneous if all the Gamma distributions have the same scale factor, but harder otherwise.  Could you post information about the specific sum you're working on?
(1) What would you do if you were just to use all seven body parameters?  (2) Why would you change your approach when restricting to the three components?
For 2-way cross-classification, I would have thought of [*ordinal quasi-symmetry* model](http://www.sciencedirect.com/science/article/pii/S0167715206002914) (which account for pairing, unlike usual Pearson $\\chi^2$). I also gave an [example with R](http://stats.stackexchange.com/a/5258/930). In case the OP cannot access Psychometrika, John Uebersax provide a good overview of [polychoric correlation](http://www.john-uebersax.com/stat/tetra.htm).
part of this answer will depend on how much data you actually have, but in principle yes you can fit nine 2 level factors to a binary response. As for "blocks", do you mean random effects? If you only have two levels for these predictors I would suggest not using them as random effetcs.
No, it doesn't work like that. If the two groups are independent, then read this link http://www.kean.edu/~fosborne/bstat/06d2pop.html
1) Good point - changed to A,a instead of H,h. 2) When significantly more matches are won by the respective home team - then this is already a valid result about whether or not this effect exists. Of course one can always choose a more complex way to factor in further aspects. Also the question addresses a theoretical problem not just the practical question.
Thanks Andreas. I fixed it. You are right of course, and that is what I entered into R. I am not sure how I copy and pasted something totally different into this answer, but it is fixed now.
@Dilip Probably my translation to English was unsuccessful. I'll try to explain it. What I want is the probability of at least 6 of the 13 units to have time of life less than 1.10.
@Damien Sounds like you could do an _intervention analysis_. One option would be to use a standard regression model with a binary indicator variable as an independent variable (switched "on" for the period in which the event occurred) and ARIMA errors. Very similar to the ARIMAX / transfer function models mentioned by others, but easier to interpret in terms of effects of the input onto the output. Rob Hyndman has a blogpost on this [here](http://www.r-bloggers.com/the-arimax-model-muddle/).
This question is complicated by the fact that some of the variables have so many levels.  Other than that the range is 5-100, what is the general distribution?  Is there any way of re-conceptualising or reshaping the data? With "many" possible explanatory variables including some that have 100 levels, even 50000 datapoints starts looking quite small.
See http://stackoverflow.com/questions/2492947/boxplot-in-r-showing-the-mean for solutions using R
How about a uniform distribution over the interval [0,1]?  Or, more generally, a beta distribution?
Well said Peter.  Interpretation of "main effects" is not usually fruitful in the presence in interactions.  One can form hypotheses or estimands of interest and then test or estimate those.  These can be stated in terms of comparisons that are meaningful to the researcher.  For example: Is there a difference between A=1 and A=0 for either level of B (a 2 d.f. test)?  Is there a difference between A=1 and A=0 when B=0 (1 d.f. test)?
It would make you answer self contained if you would summarize the main points of your blog post that relate to the OPs question. Also, by RQS and RSQ do you mean the same thing and what exactly do the abbreviations stand for?
I accepted @RyanBower's answer since it arrived first, but both of these have been really helpful.
I created a tag "chemometrics" for your question.
There isn't enough information here for this Q to really be answerable. What do you mean "compare them"? Do you want to look at differences in means from 1 sample to another? What kind of graph are you using now (eg, box plots, scatterplots, etc)?
@cardinal i removed the abbreviations for you, i'll update the question to make it more clear
Okay, thanks again!
+1 for idea of using groups. It stil seems a bit complicated, since it requires some comfortability in algebraic methods, when Dmitrij's idea really is more elementary. On the other hand this gives a deeper understanding of the topic. I also corrected some small typos and changed formatting, I hope you do not mind.
+1 Thorough and nicely illustrated.
@Pr0no  How can you tell that the number .258 listed below Durbin-Watson is the value of the test statistic and not the p-value?  It doesn't look obvious to me.  The p-value would seem more likely to me as it would be interpretable by the user whereas the value of the test statistic would not.
Don't see why it would be impossible. With a large-enough sample size, any effect will be significantly different from 0.
+1 Very nice! A derivation from basic principles is always satisfying and the graphics help the reader to apprehend instantly what you are doing.
Cluster-then-train-a-classifier do make sense. Another approach which also make sense is to cluster up new data - assign the new objects to old clusters by the cluster analysis itself.
You didn't explain how information-losing cutoffs would help.  And note that if a cutoff for a variable exists it must be a function of the values of the other variables.
A percentage without an absolute magnitude or a single magnitude without an overall measure of scale both lack part of the picture, so in that respect neither is an ideal solution. I don't see anything wrong with percentages if they convey better what it is you wish to say or interpret.
Please tell us what "w.p." means.  Is it "with probability"?
Given that the expected value reported by the `ape` library is $-1/(N-1)$ (see a recent question on the [GIS site](http://gis.stackexchange.com/a/99667/751)), I suspect they are converting the weights matrix to be row standardized under the hood, (or just mis-reporting the expected value). This is only the expected value in the case the spatial weights matrix sums to 1.
Does the remark "There is also a Volume 2B but it is Bayesian Inference." mean that Volume 2B should not be included in the description "comprehensive and authoratative (sic) reference"? because it does not have one or both of these properties, or it should not be included because it deals with Bayesian Inference rather than with purely frequentist approaches as Volumes 1 and 2A presumably do?
Please see FAQ on how to ask the questions. Also example how Chebyshev inequality helps you would be very welcome.
@Henry note the "Note" at the bottom that apologizes that the histograms are not to relative scale.
There's a subtlety here, Tim, that is obscured by Wikipedia's loose language: $W$ itself will not converge to any distribution at all.  In order to converge, it has to be *standardized*, requiring an (infinite) sequence of adjustments. For an illustration and explanation of the distinction please see my answer to a related question at http://stats.stackexchange.com/questions/49123/central-limit-theorem-and-the-law-of-large-numbers/49127#49127.
Thanks for the informative answer, and the terminological correction!
@Nick: I read that, and I'm still confused. Elaborated above.
@jbowman I agree, in this case since it is uniform, it doesn't really matter. But the question in hand needs distribution of $\\theta|Y$. And I can't really write $\\theta|X \\approx \\theta|Y$. The answer is same if I use $X$ instead of $Y$, as it would take maximum of $Xi$. But the real problem is in understanding **point estimate for $\\theta$ based on the median of posterior**. What does it really mean?
Or you could just threshold the process $(z_i)$. The relationship between the autocorrelation of $(z_i)$ and that of the binary thresholded process is a little more direct.
A related question appears at http://stats.stackexchange.com/questions/41438/covariance-of-inid-order-statistics.  Although it does not answer this one, it might provide some insight.  (The distributions of the min and max are easy to find, btw.)
I'm trying to understand the question. Are you asking that how you can write down the likelihood of the bigger matrix when each of its rows is characterized by a separate Gaussian mixture model?
I am not seeing how to retrieve something like a_uncert or b_uncert from freude's answer. Can anyone clarify that?
**Hint:** Algebraically manipulate $0 = \\text{Cov}(u,v)$. You should expect to use the double-angle formulae $\\cos(2\\theta)=\\cos^2(\\theta)-\\sin^2(\\theta)$ and $\\sin(2\\theta) = 2\\sin(\\theta)\\cos(\\theta)$.
I think you would need to adapt the idea of TOST. The most straightforward way would be to embed the exponential family in a larger family (of, say, two, three, or four parameters) that captures the likely deviations from exponentiality you might expect and then test those parameters for largish deviations from the values that would correspond to an exponential distribution. As you can see, answering this kind of question can get pretty involved: it's much easier to establish evidence that a hypothesis is *not* the case than it is to build strong evidence that it *is* the case.
@O_Devinyak Your comment that ‘[i]f ANOVA gives you p>0.05, then all post-hoc tests are redundant and are non-significant regardless of their p-values’ is not true. Most post-hoc tests (including Tukey’s HSD) do *not* depend on the global test being significant. (And conditioning on the global test being significant actually changes the statistical properties of the post-hoc tests.) See http://stats.stackexchange.com/questions/9751/do-we-need-a-global-test-before-post-hoc-tests
Rather than decision boundaries you will probably find more utility in considering posterior probabilities of class membership.  This can be done with fewer assumptions using polytomous (multinomial) logistic regression but can also be done with LDA (posterior probabilities).
As far as the possible/impossible issue goes, you might want to settle for practicability instead. It's certainly possible to count the coins in active circulation during a short period of time: perform a [mark and recapture experiment](http://en.wikipedia.org/wiki/Mark_and_recapture). The number of coins to mark would have to be enormous, though, making such an approach theoretically *possible* but not *practicable*.
The theory is that linear combinations of normals are normal. That's basically what you are doing here.
While the data is clearly skewed, with a sample size of 60 (probably 20 per group) the CLT may give a good enough approximation (multiply the p-value by 10 and it will still be significant at the 0.05 level).  It would be interesting to look at permutation and/or bootstrap tests comparing the means and the medians of the groups (even though the KW test is not technically a test of medians either).
I have to disagree.  For someone who has already said s/he's feeling overwhelmed by the information out there, Harrell with his level of detail will surely make things worse.
(+1) Nice response. The use of several classifiers on the same dataset that yields to over-optimistic measure of test accuracy have been discussed in [Optimal classifier selection and negative bias in error rate estimation: an empirical study on high-dimensional prediction](http://bit.ly/rLsCTD), BMC MRM 2009, 9:85 (with some PDF slides [here](http://www.wias-berlin.de/workshops/validation2010/workshop_slides/talk_boulesteix.pdf)), with other discussion in [Multiple-rule bias in the comparison of classification rules](http://bit.ly/sdF1tr) (Yousefi et al., Bioinformatics 2011, 27(12):1675).
I've changed the formatting for better readability, check to make sure I did not change the original meaning.
scale here is very important. Back propagation can be paralleled massively (indeed, if the number of variable is large it is close to embarrassingly parallel) in a way that GA isn't.
Do you know which predictors will be missing?
@Shuaiyuancn:, What do you mean the probability of each outcome type is unknown? Does it change over time? If not, then you can infer the probabilities by observing a large number of outcomes. Your expression in the comment above is correct in spirit, so long as your corresponding probabilities are correct (0.95, 0.025, 0.025) but I have no way of knowing whether they are or not.
Please don't mind me pinging (one and only time) to see if you have missed the follow up question. Thanks! (And sorry for bothering if you chose not to elaborate).
Hmmm. I would think that most mathematicians use the MVT to approximate an integral as a rectangle.
P.S. The technique being described there is for a two-sample test; the approach you take with a one sample test has to be a bit different.  When the data is continuous you need to use a sign test as indicated by caracal.  I'm not sure a sign test works as expected when many of your values are 0 which is why I suggested the permutation approach I provide below.
@whuber, I will admit 2 and 3 are based on the assumption that 1 is true, and also that the parenthetical about spreadsheets is an opinion (but does "to the person" in Latin apply to computer programs, but then I would be surprised if there was a Latin word/phrase for computer program).  The fact that the standard definition of SD does not allow a negative number is fact rather than opinion.  I thought 3 would be helpful (assuming 1) pointing the user in the direction of math errors.
Here's a simple equation for when the initial correlations are 0. Let's say I made the initial variances a = 6, b = 2, c = 1.5. The proportion of shared variance calculation is still the same for x where the proportion shared variance is var(a)/(var(a) + var(b)) or 6/8. For y it would be 2/3.5. To find out their correlation you have to get the geometric mean of those shared variances sqrt(2/8 * 2/3.5). (You'd have to correct the initial shared variances by the covariance if the a, b, c values were correlated.)
I think the null is $H_0: a_1+a_3=0$.
Ok, I am a bit confused as to what you are trying to calculate. It seems like you are tring to estimate the following limit: $\\lim\\limits_{n\\rightarrow \\infty} \\frac{1}{n}\\sum\\limits_{i=1}^{n-1} \\frac{N_{f(x_{i+1}-x_i)}}{N_{g(x_{i+1}-x_i)}}$ Where $N_{a(i)}$ is the number of counts in bin i for the random variable with distribution a.
The tone of the question does not invite constructive answers. Please consider rewording your question.
@Whuber, is his solution correct?
Sorry I can't understand your models. Formulae such as `A+B+B` don't make much sense in the usual statistical model-description language as repeated terms would simply be dropped. Nor do I understand how e.g. `A  A+B  A+B+C   A+B+C+D` can be one single model.
It is quite often the problem, that the outlier component has a large share of the data. But even when this is not the case, the problem occurs. And the difference is to big to be numerical only. Here for example the log likelihood from one run -583.3709 -420.7938 -407.7895 -416.2775
@JonBonJovi, I don't know exactly what's going on with your numbers it depends on a lot of things about your data and how your software works behind the scenes that I can't diagnose from here.  Note that MSE is an estimate of the residual *variance*, not the SD.
I cross-validated over the options. Check out the papers by Gloor GB. All of his are great. "Validation of coevolving residue algorithms via pipeline sensitivity analysis: ELSC and OMES and ZNMI, oh my!" also was a primary influence on the paper.
m-out-of-n bootstrap is also commonly referred to as subsampling, as the title of the book by Politis, Romano and Wolf goes.
Probably I miss something, but if the reward for every potential action is known, then why do you need multi-armed bandits? The optimal policy becomes trivial, just pick the best action...
oh by the way - I hesitated as to whether to put this here or on stack overflow - moderator pls advise. Didn't want to post it to both.
+1 especially for the hint.
I suspect the point of this question is to determine an appropriate standard deviation for the distribution. :-)
If there is something available in R? I can reverse engineer and implement Java. That helps me in validating Java output with R output to make sure code I have is valid.
The p-value is not the probability that we are wrong when we reject the null hypothesis, as that also depends on H1 which does not enter into the calculation of the p-value (well illustrated by http://i.stack.imgur.com/tStr4.png - the probability that H0 is wrong and that the sun has exploded is rather less than p = 1/36).
Do you have any a priori hypothesis about the relative strength of different approaches (e.g. approach1>approach2>approach3)? This is crucial to answer your questions.
The real issue comes not because some observed cells are 0 but because some columns are all-zero. This makes the expected values in that column zero, which makes the contribution to chisquare of each cell in that column, $(O_i-E_i)^2/E_i = 0/0$. You can't compute a chi-square in that situation; in R, $0/0$ is `NaN`, and a sum that includes a `NaN` is a `NaN` - it needn't trap it because it returns the 'right' answer by default. Before calling the scipy function, check your row and column totals are all > 0.
Your phrasing here: "* [4,0,0,0...] should be as far away as [32,0,0,0,..] but with lower significance*" suggests that your notion of distance or significance will have to change for it to make sense. I don't see how the two can be consistent as it stands.
Great reasoning! :) This is elegant. :)
An additional opportunity/complication you don't mention is that you could also use the sample variances to estimate c (and doubtless would get a different estimate to that which comes from the sample means).
This is a good idea only if his observation data has high granularity (ie- minutes). If he only has data by hours (which the question seems to imply), I'm not so sure a continuous Markov Chain would be appropriate. In the specific case of a Birth-Death process, it wouldn't work because I'm sure the differences between hours is more than 1. I'm not too refreshed on Markov Chains, however, so feel free to correct me :)
Certainly the prior matters quite a bit when analyzing large datasets, for most interesting modern problems. True, we have asymptotic results when $n \\to \\infty$ and $p \\ll n$, but typically people with big datasets ask big questions so that in some sense $p \\to \\infty$ at a rate comparable to $n$, and here an appropriate prior is going to act like a regularizer and will matter a great deal.
The answers can be found at http://stats.stackexchange.com/questions/9131, http://stats.stackexchange.com/questions/44845, and http://stats.stackexchange.com/questions/16493 (even though they deal with prediction intervals). If that's not enough, more would likely turn up in a search for [regression prediction interval](http://stats.stackexchange.com/search?tab=votes&q=prediction%20interval%20regression). As far as the second part goes, it comes down to the difference between a standard error and a standard deviation: the standard errors vary with values of the IVs but the SDs do not.
Are the *X* stimuli different in an ordered way, as might befit penalized regression, or are they more nominal, as would suit an ANOVA? It sounds like you could use a multilevel model to separate within-subject random effects on reaction time from group-level effects of your stimuli, which I think (but I'm not sure that) you'd want to treat as fixed effects. About *p* values vs. effect sizes: the difference is mostly in emphasis on **(A)** inference regarding evidence against the null hypotheses in the population and **(B)** your sample's best estimates of differences (if any) among groups...
@Manuel I think I came up with an answer to my problem. I'm not a stats guy and looking back on it, I didn't frame the question very well.
@Fred Sorry I missed the detail. See added comments.
Several questions here, to the extent that this is likely to be declared a duplicate. I would start here http://stats.stackexchange.com/questions/2492/is-normality-testing-essentially-useless as the real question to many is not can you test for this, but is that a good idea?
Is the KL minimization you refer to is done while containing p? (Otherwise, the minimizer will be trivially q). If so- KL minimization can be seen as an orthogonal projection, under KL norm.
It depends on the community. For instance, Excel calls it a "high-low-close" plot because such things are used to chart prices of assets in markets.
@Baumann my original plan was to run a logistic regression and identify those variables that are best to predict the criteria. that didn't work for several reasons - my plan now is to do this by using correlation and/or parametric-tests. does that make sense?
@onestop I am not sure I understand what you say when you write that it works under a "rather restrictive normality assumption". What do you mean by that ? I found very general the idea of projecting a "new" observation on the centroid difference after normalizing everything with the covariance geometry don't you ?
@Jason from the original post "I also want to make clear that I do not think that this is a good system."  However, it is a good example of how to fit a support vector machine to a time series.
For the probability that $Y_i = 1$, you gave the Bernoulli likelihood: $\\pi^{Y_i} (1-\\pi)^{1- Y_i}$. The number of 'successes' of $n$ many Bernoulli trials is given by a Binomial distribution.