tttjjj/elt_bench huggingface dataset:
It seems like you're not asking the question that you really want answered.  If you want to be 95% sure that the probability that you have a bad apple in the bunch is zero, then your sample size should be 9,500.  Another question you might be able to answer might be how many apples do you need in order to be sure that the proportion of good apples is above 95% with 95% confidence if all of the apples you sample are good.  For that the answer would be 59, because $0.95^{59} < 0.05$.
-segmented- will sort of force a breakpoint based on how many you've chosen. I'd check models (using AIC) between 0,1,2 breaks to make sure the number of breakpoints chosen is justified. (you'll notice the breaks above U1.x U2.x don't have p-values)
Cluster analysis might not need your criterias. It may return anything. Did you consider setting up thresholds (high/low) to define these four groups manually?
@ttnphns - Thanks for your insight. I guess I wouldn't be so worried about the result given by `svd` if it weren't for the different shape of the eigenvalues. The result is obviously giving more importance to the trailing eigenvalues than it should.
It is not entirely clear to me whether you are looking for a trend (which you can assess with `summary(lm(Pop~Year,DF))`) or for structural breaks. Could you clarify your question? The `strucchange` R package offers ways to assess structural breaks, if that is what you are looking for: http://cran.r-project.org/web/packages/strucchange/index.html
@drknexus Adding up to 100 is an issue which is mitigated to some extent if you do online surveys where the software can check if the total adds up to 100. Not being able to tell the difference between 90% and 95% is measurement error and that can be dealt with if necessary. The question form I suggested is one of the standard question types used in market research but has not been used much in the past because of the issue about totals that you mentioned. I feel that the issue of totals is much less pertinent in online surveys.
+6 Amazing post: informative, insightful, and fun to read.
@StéphaneLaurent, OK, I see. But maybe in the first example you could make `tau` a hyperprior...
Hmm, I mentioned $T \\in [0,5]$ because the subject was only susceptible after being born ($t=0$). Am I interpreting left-censorship wrong? Is there no natural lower-bound?
+1. A note on your last points -- It seems most likely that the 11.5 as reported would be rounded ... E.g. A chi-squared statistic of 11.53 would return a p-value of 0.2411. (But the OP does say 11.50...)
For homicide rates, 100,000 is essentially the smallest number needed to not report the rate in decimals.
Are you provided with any form of sampling distribution?
@Chris You would have to make some more assumptions.  As it is now there are a multiplicity of valid explanations.  Here’s a silly one: Are the individual’s visits independent of yours?  If not, maybe he will only visit when you visit (he looks for your car outside everyday), but tosses a coin (with probability 0.9) before deciding whether to go inside.
As to the choice among GEE, clogit and mixed GLM for binary data, some people are in favor of clogit since some of the consistency properties break down, especially with small within-subject sample size; some prefer GEE to obtain population average estimate as there is attenuation of the effect of the covariates for subject specific model (e.g. mixed GLM). You can have your own choice based on your data format and analysis goals.
@VitalStatistix, thanks i guess so if not whole of my code will give wrong result
@dcl Added one more source that may help.
The matching is a *crucial* fact, Martin, because it makes available a wider set of estimators of the performance differential, suggesting your question could be answered by *ignoring it* and proposing another estimator and analyzing its properties. Would you please edit the question to state that the data are matched (and to explain how that matching occurs)? I would also like to point out that 10 values is nowhere near "asymptotic"! Moreover, your reference is wrong: the ratio is *not* asymptotically Cauchy: see http://en.wikipedia.org/wiki/Ratio_distribution#Gaussian_ratio_distribution.
If the columns of matrix A represent attributes and the rows mean a observation vector the first calculation is correct and if columns represent an observation and the row represents attributes the second method is correct?
@FrankHarrell I've broken this discussion out [into a new question](http://stats.stackexchange.com/questions/18745/if-a-factor-variable-is-to-be-dropped-in-model-selection-should-all-levels-be-d).
Degrees of freedom of what? You said "degrees of freedom for these data", but the *data* d.f. is $n$, not $n-k-1$. Do you mean the d.f. for the error term? Is this an ordinary regression model or something a bit more complicated?
By the way, I can find "sparse PCA" methods -- can any of these be used to find sparse _minimum_ rather than maximum eigenvalues?
This looks like a reasonable answer to me.  I would also like to add, it is likely they look at the distribution of views over time to make sure that there is no systematic trend that a simple bot might create.  Although bots could certainly be made to appear to randomly watch a video, it would be an easy first step to remove "stupid" bots who systemically view the video.
Do you mean profit or revenue ? If you mean profit there should be no additional costs. If you mean revenue, I see no problem on reporting back to a decision layer whether one group outperforms the other one significantly (regarding revenue). Calculating the costs (which includes you performing the analysis ;)) is not your job, isn't it ?
Perfect. I figured that part would be relatively straightforward, and I'm glad that the preprocessing can make this a quick calculation.
Sarah, welcome to the site. You can also ask this question on SEMNET mailing list.
(+1) Welcome to 3K.
@ttnphns I edit the question to clarify what I want regarding to the matching on the shape of the curve.
As  for Michael's comment, note that Poisson with parameter gamma distributed __is__ the negative binomial model ....  It could be easier, speaking of programming, to use the negative binomial directly.
The trick applies to non-integer powers, but the proof is a little different.
I think perhaps the difference is what margins are being conditioned on.
Indeed, $P(X_1>X_2 , X_1>X_3,... , X_1>X_n)=P(X_1 \\geq X_i\\, \\forall i \\in \\{2, ..., n\\})$. This can be reduced to $P(X_1 \\geq \\max_{i=2}^n X_i)$ which is analysed in one of your links.
I agree with this answer : )
+1 for Jaynes, Cover and Thomas and Vapnik; after a book like Barber (or Bishop or Murphy) it is probably better to concentrate on books that go into depth of one particular idea rather than breadth.
Just  to be clear, I'm mainly interested in the overall question of how to treat the data, rather than the intricacies of comparing  gene -expression.  He has other data on protein levels, length of roots etc that he'd also like to look at.
No, I don't know any. There are way too many papers on k-means, unfortunately, despite it being actually a quite boring optimization task. And too many just repeat things they read somewhere, often without giving the sources themselves.
+1, this was my first thought as well. I like the other suggestions, but this seems the best to me.
The citation is for: 

Kruschke, J. K., Aguinis, H., & Joo, H. (2012). The time has come: Bayesian methods for data analysis in the organizational sciences.

But Kruschke has used it before in: (2010) Bayesian data analysis and (2010) What to believe: Bayesian methods for data analysis.
Thanks! I'm really glad that this solves your problem. As for expertise, I follow CV daily, and it's always an education in how much I have yet to learn. On the other hand, I'm probably punching above the weight class for BA's in political science... ;)
To add to your example, if $A = B = 0$ instead, then $E = F = 0$ _are_ independent, and thus the answer to the OP's question "Are $E$ and $F$ necessarily independent?" is that $E$ and $F$ are _not_ necessarily independent; they may be dependent or independent as illustrated by the examples.
ASA implicitely gives a low probability
+1 for multicore - very useful indeed
It's actually already answered on StackOverflow, several times. For example, see [here](http://stackoverflow.com/questions/24173468/r-print-equation-of-linear-regression-on-the-plot-itself). If it's migrated there it will immediately close as duplicate.
Isn't this exactly the Girl comparison in the Facebook legend, and movie? Would the chess ranking algorithm work here with a series of random pairings?
@singelton Yes, but in view of coherence with the answer it would be better to denote $\\theta_1$ instead of $\\theta$.
@ Mpiktas, Do you also know some interpretation on Dlog - log ... ? Is that similar to the interpretation of log-log?
That's pretty close. The problem is that students end up in state 4 at around the point at which the chain becomes time independent, and not too much before. I haven't looked at this problem in a while, and this has just occurred to me.
Hi @mfernan5, I guess I need a little help understanding your research question.  If the question is pre/post changes in scoring, I am not sure why differences in teacher + student cohorts is important.  You say some groups of students did drastically different; does this mean they changed on the pre/post less than the other groups, or that their pre-scores were lower? Rasch (and everything else) is quite sensitive to sample size, could you please share the sample sizes for each teacher-student cohort and let us know the size of the standard errors reported.   Thanks!
By prognostic I mean, "able to predict a patient's outcome."  In my example, it could be the model's ability to predict probability of 5-year overall survival. 

Yes, the model was trained on a broad population of breast cancer patients, but the researchers then test it on a bunch of sub-populations and state conclusions like "the model performed well with X types of patients but not Y types."  My gut tells me you can't draw conclusions like that if you trained the model on a broad population.
Are you talking about the effect of zero cells on the magnitude of the coefficient or on the test of significance? We know that the max possible value of kappa is very distribution-dependent - see wikipedia and [this](http://www-users.york.ac.uk/~mb55/msc/clinimet/week4/kappash2.pdf). So there is no limitations, but issues of interpretation of magnitude. As for significance testing, zero cells / small sample is a problem; however, you can compute the exact p-value instead of the asymptotic-based.
@Mermex, you can accept the answer if you think it answered your question. See [faq](http://stats.stackexchange.com/faq).
I have not yet obtained a concise answer how to proceed with the two sets of data collected (baseline emotions from Mood Questionnaire) from the pre-test AND emotional responses (subjective feelings from Self Assessment Manikin questionnaire). The issue is that there are two different kinds of questionnaires for pre-test and post-test, geared to capture initial and final "arousal + valence" - emotional responses. Any help would be appreciated.
Breslow-Day's null hypothesis is "homogeneous odds ratios across strata". Since a p value > 0.05 does not imply the null to be true, you can't assume that the odds ratios are equal.
@chl: i edited the question for more details. thanks for your help in advance.
Are you asking about [*validity*](http://stats.stackexchange.com/a/82090/919) or about [*significance*](http://stats.stackexchange.com/search?q=significance+is%3Aquestion)?  Those two words have different meanings.
(+1) Well, the Gaussian distribution is supported, you just aren't allowed to call it the Gaussian distribution :) I also made a minor edit - hopefully you don't mind
In some situations an F test or $\\chi^2$ test will work as expected and in others they won't, depending on how the data are assumed to depart from independence.  In what way, precisely, do you suppose your two samples are dependent?
Without being familiar with the SVM package you are using I can't give any better information than the fminsearch help message.  I use my own matlab software, which has model selection built in, but unfortunately has no manual, see http://theoval.cmp.uea.ac.uk/projects/gkm/
Some corrections are needed to stave of possible confusion: first, $e_j^\\prime$ is an eigenvector, not an eigenvalue. Second, it does not assert any relationship at all between $X_1$ and $X_2$: it is literally just a linear combination of the two. FWIW, googling "linear regression test contrast" turns up [the Wikipedia article on the subject](http://en.wikipedia.org/wiki/Contrast_%28statistics%29) right away.
Thanks! That is helpful. But what about interaction terms e.g. `cond x subject`? should they included or excluded from the regression? The funny thing is matlab reports `main effect of subject p<0.00001` - do I just discard this as irrelevant, or should I somehow stop matlab from making this contrast?
@JacobMick: My work is very simple. Given a image. How to separate that image into small region with center at pixel x. So gaussian kernek K(x) will divide image into small image with size is $\\sigma$. Hence, to separate for whole image, we only do convolution. Do you have any kernel can do it
Sorry, but -1 from me. This is true in general, but the residuals won't be if we have an inflation of zeros. I think answers shouldn't just cover generalities but adress the specific issue.
I tried to clarify the formula. To solve your speed problem, you need to put your probabilities in a **tree**. That will do wonders!! This is an idea I had some time ago to perform massive multiple replacement, we can talk the detail some other place if you want. I you have space for the words, you should have space for the the nodes of the tree. Computing these scores will be the ratio of the prob of the children node to the prob of the current node which you can probably compute between key strokes. And you can merge triplets and singlets in the same tree, their probabilities will be scaled.
Generate a random float _x_ in the interval (0,1) and perform a binary search of the cumulative fitness array, finding the first entry that equals or exceeds _x_.  Its index denotes the chosen fitness.
Ha, thanks? :-)
It doesn't really tell you anything.  You should really post the means you have in your question.
@Peter Formal hypothesis testing of normality to assess whether to assume normality in some other procedure (not just Shapiro-Wilk, *per se*). On the other hand, I do use Q-Q plots, which indicate something of the extent of non-normality (i.e. something closer to "effect size") and give a more useful indication (i.e. is somewhat more related to the question of interest in that circumstance ... which is how much effect the non-normality will have).
$2^{100}\\approx 1.27\\times 10^{30}$. It looks like approaches that rely on going through all subsets are out of the question. This related recent question may be helpful: http://stats.stackexchange.com/questions/47399/how-to-use-ols-or-nls-on-very-small-sample-size
@wannik If your training and test set are random samples from the actual data, then they must have identical distributions. Almost every classifier we use expects data to be of this form.

However, the situation you describe is a fairly common scenario. It is hard to predict the behaviour of the classifier in this situation.

Generally, 1] Use plain classifier, if it works then great, 2] If not, do you know the class proportion in test apriori? If yes, then use transduction SVM 3] If not, then use the same approach described in the original answer (importance weights).
Does it really make sense to test all pairwise comparisons? Physiologically would you expect arbitrary effects for different doses? It's not my area of expertise but I would rather expect some monotone function (increasingly strong effect, perhaps with some threshold or leveling off).
Absolutely.  You appear to have a one-dimensional situation that is almost exactly like a running example in the Diggle & Ribeiro manual for geoRglm (malaria cases in Gambia, with proximity to marshes, etc., as covariates). The main complication is handling the change of support, but that wouldn't really affect the prediction: it would primarily affect the estimation of the variogram. See http://www.ncbi.nlm.nih.gov/pmc/articles/PMC2995922/ for some theory and similar examples ("binomial kriging" of disease cases).
A pedantic comment: you meant the coefficient estimates, and you meant those of the non-constant explanatory variables. The intercept estimate is different between the full regression and the orthogonalized regression, but it is probably of little importance.
I cross-posted this in Statalist: http://www.statalist.org/forums/forum/general-stata-discussion/general/32039-is-it-possible-to-fit-a-graded-response-model-in-stata
Also what are you going to do when a word has a zero count for a day?  Infinite % increase?
I'd rephrase that "the most popular statistical tool in bioinformatics"... Bioinformaticians doing microarray analysis use it extensively, yes. But bioinformatics is not limited to that ;)
Sorry, I didn't mean to seem to suggest that. There are two considerations: (1) Orthogonality ensures the estimate of the treatment effect is independent of the subject effects - it's the same whether you include subjects or not, & the only issue's getting a better estimate of its standard error; & (2) including subjects in the model may increase prediction error (though orthogonal designs can typically get away with fewer observations per regression degree of freedom than rules of thumb for observational studies suggest is necessary). The first consideration is the relevant one for people ...
Apparently yes: http://onlinelibrary.wiley.com/doi/10.1002/0470011815.b2a13058/abstract

I assume you did do a Google search on "multivariate Weibull". It would make it easier to help you if you told us specifically what about the Google results was not helpful to you or what you are looking for in addition.
Perfect! So clear! Thank you so much for your replies and your patience with a beginner!
+1 for the stand against stepwise regression. @Dimitrie, see 'Modern Epidemiology' by Rothman, Greenland and Lash for an approachable discussion of potential variable selection techniques.
@varty That's an excellent question.  If one merely collects, say, centroids of houses, the project looks hopeless.  If the houses are represented as polygons there's a bit more hope, but it appears that having data *representing the streets* will be key, for then it's possible to distinguish housing clusters at the ends of streets from identical-looking clusters elsewhere.  For an example, see http://www.topoquest.com/map.php?lat=42.13070&lon=-121.82028&datum=nad83&zoom=4 .
Induction vs Deduction?
$p_0$ is the restricted maximum likelihood estimate of $p$... which is the unknown parameter of interest. The null hypothesis has restrictions about what it can be... hint it's just one point.
Actualy the roots should not be on the unit circle. If the roots are inside the unit circle, the solution is stationary, but not invertible.
1) no i'm not interested in the relationship of the dv. 2) all of the variables are not categorical
I'm a bit confused. Is stimulated=treated and unstimulated=untreated? Don't you want to use both sets of data to define a threshold? Then you'll get some estimate of false positives/false negatives.
@cardinal: page 12 (I updated the question).
One of the assumptions of RD is that the other variables are smooth functions of the assignment variable conditional on treatment. I would worry that would not be the case with county and municipality data, so that would favor PS matching. In your example, there's also clearly no selection into treatment, so that would favor matching.
Your question says "what do I do with all this information". You may get a better response from this site if you split your question into separate more manageable and specific questions. I recommend that you do some more general reading on meta analysis and medical statistics. For what it's worth, here are a few resources that I prepared on meta-analysis: http://jeromyanglim.blogspot.com/2009/12/meta-analysis-tips-resources-and.html
+1 This reply shines because it offers *quantitative* reasoning in its support.
@user777 https://d396qusza40orc.cloudfront.net/bioinfomethods1%2Flabs%2FCoursera_BioinfoMethods-I_Lab02.pdf Read the first line. There, the nr stands for nonreductive database(As told by the instructor)
I would have thought you can't do much better than taking the maximum of the sum.  i.e. $P(X>E[X]+tn)\\geq P(X=k)$ where $k$ is the smallest integer above $E[X]+tn$ (for $t>0$)
@user603 Yes. The first sentence in your post looks so much better! now I am more inclined to listen to your argument which is sensible. I was so turned off by that first sentence that I didn't read the rest of it at first.
This is very broad....Perhaps too broad. Can you narrow it down? What sort of models? What sort of distributions?
A comment: and what about NOT scaling/normalizing?
a related question on academia.se provides some insight: http://academia.stackexchange.com/questions/10660/what-does-principled-approach-mean-in-papers
That might have worked, but on some thinking I found a (hopefully) better idea.
Some of the density based are grid-based. So these probably don't make a lot of sense for you. But those that use distances for density estimation should work.
(+1) That's an interesting response, thank you very much. My main concern at the time of this writing was that cutoff values will be somewhat "interdependent". But I'll check Pepe's book (found some handouts [here](http://labs.fhcrc.org/pepe/fdacourse/) in the meantime).
Closing, because this question is either empty (interpreted as written) or it covers previous ground.
You seem to be confused about several things. Your title says 'always fail to reject' ... but your output has 6 out of 7 rejections? How do you account for that rather extreme mismatch between what you show us and how you describe it? If you do understand the problem there, *could you change your title please*? It sounds like you need to review the logic of hypothesis testing. You can't have $p\\neq\\frac{1}{2}$ in the null -- for starters, how would you work out the null distribution of the test statistic?
Because this is a homework/self-study question, it should have the `self-study` tag. See http://stats.stackexchange.com/tags/self-study/info
Use `sandwich` in R. Or `robust` option in Stata. A year and a half of waiting time is definitely worth the price of Stata :).
As I understand it, you are in favour of keeping it. But when analysing (loglinear regression) this category is always contrary to theoretical expectations and the trend between the other 3 categories. Can I be certain this is not due to the sparseness, and these result are trustworthy?
Could you give a couple of examples of what your code does?
I am sure the historical data is 'de-biased' I have looked up importance sampling and it appears to be just what I require. Thank you for your help.
Constraints that occur with probability zero are hard to satisfy. ;-) For the specific example at hand, an appropriate shift and dilation easily achieves the *stated* goals, though one might want to analyze a bit deeper to look at how the distribution of the data is perturbed by such an operation.
Welcome to Cross Validated. What exactly are you stuck on and what work have you done so far to solve the problem?
Do you want a global test for the three groups with the null hypothesis that the three medians are equal? Or do you want pairwise comparisons? All the tree pairs (control-prior, control-after, prior-after)? Will you draw the same conclusion if one or an other pairwise comparison shows a difference?
@Nick Sabbe, I have, and am trying to wrap my head around them. I was wondering if there are any nice intuitive interpretations for them.
@whuber i removed the section on the mode since it wasn't even part of the question anyway