tttjjj/elt_bench huggingface dataset:
@Drew75 That's a little oversimplified. It is perfectly possible to define and calculate $R^2$ for logistic regression -- but it can be done in several different ways and $R^2$ is not nearly so central or natural a figure of merit for logistic regression as it is for linear regression. For a well-informed defence of standard $R^2$ across a range of models, see e.g. http://www.ncbi.nlm.nih.gov/pubmed/10861777
If your goal is simply to maximize the probability of success for each event, it seems to me that the optimal strategy would always be to schedule everything in the interval with the highest estimated probability of success (which wouldn't necessarily be the highest empirical probability, but could depend on how you compute your estimate). Is there some other metric you're interested in that would lead to a different solution? Or maybe some load balancing requirement?
Yes. I have explained the abbreviation, added some information and a link and corrected two typos in my original.
@user777 You're on a good track. There's a stupid (not clever) procedure but in extreme cases it can be effective: assume the worst and suppose there were only two measurements in each group and conservatively replace the smaller SD by the larger one.  Also assume the measurements are iid Normal. If the resulting t-test identifies a difference, you have made some progress. If it is inconclusive, the results are inconclusive. Without some kind of distributional assumption (like normality), nothing can be done.
The distribution of age is certainly not log normal.
I thought as much. It is not exactly a graph question but a question about sampling from different populations. Considering that a t-test should probably be enough, however I wanted to get an other opinion. Thanks.
you could probably get a better answer if you provided more details about your experimental design, research question, and statistical model.
R has a learning curve, so it depends on whether you are going to do more playing with statistics. Histograms are dangerous for assessing shape -  changing the bin or starting value can change the appearance of the distribution quite a bit. See [this thread](http://stats.stackexchange.com/questions/51718/assessing-approximate-distribution-of-data-based-on-a-histogram/51753#51753) for more about histograms. A density plot of some sort is probably what you want.
If you're not committed to a log-normal distribution for each $\\sigma_i^2$ but instead go with a normal-inverse-gamma distribution for both $\\mu_i, \\sigma_i^2$, then each $X_{ij}$ would be marginally t-distributed, which would give you what you need
`sem` is just a wrapper for `lavaan` with certain defaults. These can be specified manually for the same effect, and doing so might be wise to make sure the right settings are chosen. Also, FWIW, SEM subsumes CFA too.
@DimitriyV.Masterov: just update my post...COOP is a dummy variable.
And how does this identify a difference-in-differences parameter?
@Tim, The Wald test and interval estimates the variance and then measures the distance relative to the variance in both directions.  If the likelihood is symmetric then the distance in either direction will be about the same.  But if the likelihood is not symmetric then a value a distance in one direction can have a very different likelihood than the value the same distance in the either direction.  The likelihood ratio test would show them as very different in significance but the Wald test would give them equal significance.
I don't know if I could trust model based parameter estimates with such high influence values in the function.  You can expect your standard Newton-Raphson algorithms to give you nonsensical parameter estimates with such values of $x$ as a linear predictor in logistic regression models. Odds ratios *can* be reported as infinite valued. Furthermore, I believe you can invert the score test to obtain a valid confidence interval for the odds ratio.
I see you've lost your account -- I have merged them, but please use your original Facebook login.
@whuber, I work among folks who think extra measurements are expensive, and if I supply them a "minimum number of samples" they do not think of it as an inequality where the potential number of samples is greater than that.  They think of it as the boundary for an optimization problem of minimizing cost, and try to operate at only that value.  It is a product of my environment.
@rolando2 Could you please provide a link to (at least) one such thread?
Do you want to recognize whether each individual value is more or less frequent than expected, or just see whether there is a general difference across all 9 values?
If somebody could help me understand the recent question by me i.e. What is the unique thing about super-population for a meta-analysis? it may help. Further, it seems that one can understand the difference between two models if we think it in terms of mathematical statistics.
I think the condition of 5 is for the actual cell count and I see that one of the cell has cell count = 3. Further, the p-value calculated by the ChiSq test is based on pchisq( .. , lower.tail=F), if I understand the code correctly. Therefore, the p-value is based on one sided test, where as the CI is two sided, hence the discrepancy.
There does not appear to be enough information to answer this question, but it's not clear.  Are you saying that there is a stochastic process $Y(t)$ of losses over time?  And if we define a new process $x\\wedge Y$ to mark the times $t$ when $Y(t)\\ge x$, then $x\\wedge Y$ is found to be Poisson?  And you are then speculating that the process $x_1\\wedge Y$ for $x_1\\ge x$ will also be Poisson?
@gung: Please see my edit question
Also, is there a stepwise algorithm like step() in R that works for time series with regards to AIC or BIC? I suppose I could write my own, but it would be a pain.
Thanks, I can't believe I overlooked this!
Excellent answer. Granger causality is not causality. Christmas cards Granger-cause Christmas, but they certainly do not cause it in the philosophical/dictionary sense of the word.
Welcome to the site, @pyoi. This isn't really an answer to the OP's question, it is more of a comment. Please only use the "Your Answer" field to provide answers. I know it's frustrating, but you will be able to comment anywhere when your reputation >50. Alternatively, you could expand this to make it more of an answer. Since you are new here, you may want to take our [tour](http://stats.stackexchange.com/tour), which contains information for new users.
A *true* census or an *attempted* one?
did you ever figure out the relationship between the Brownian bridge and the plots you show in the answer you link to?
Ah, through math, problems seem so simple. :) There is a growing pile of literature on assessing tiger (and other feline species) numbers, especially in the spatial context. This is where you use spatial component of the data (where was the animal sighted?) to infer their home range, which you use to assess the actual area sampled, which is important for calculating density of animals. This (currently) takes hierarchical models and a butt load of field work. :)
@Zach: (+1) Trevor Hastie disagrees with the statement you made that "any variable with a coefficient of zero... was unimportant."  When I spoke with him this summer, he said that the variable importance measures given by other methods like random forests are much more reliable. If the OP used an elastic net penalty instead of the default of `alpha = 1`, that can be be somewhat better (see [here](http://www.stanford.edu/~hastie/Papers/elasticnet.pdf)), but my impression from talking to him was that other methods can be much more reliable. Bootstrapping is, as you said, a good idea here too.
@tracy A permutation test of the means of the data-transformed-to-ranks is just a two-sample Mann-Whitney-Wilcoxon test; if the data are reasonably close to normal the permutation test will have better power to detect differences, otherwise - e.g. if the tails are heavier - it may not. (Personally, I'd be inclined to do the full permutation test on the means as Bitwise suggests, because it's explicitly testing for a difference in means.)
Take a look here: http://xianblog.wordpress.com/2011/03/30/jaynes-back-on-tracks/. It may help...
@user2040: 1. Yes ... is there a question or is this just a remark :) ? 2. why don't you add the code to the question ? No formatting is horrible.
Because significance testing leads to so many problems let's convert to using skull and crossbones.
I'm struggling to understand why you don't think logistic regression is appropriate. The response for logistic regression is a count of $x$ successes out of $n$ trials, which is what you say you have, & the estimated coefficients give you predicted probabilities for any predictor pattern, which is what you say you want. I'm wondering if you think that because logistic regression can be used for classification (i.e. give a "label" as an "output"), that's all it does.
[Wikipedia](http://en.wikipedia.org/wiki/Exponential_distribution#Distribution_of_the_minimum_of_exponential_random_variables) has an easy answer :)
I've posted my comment and response from a moment ago` as an answer, but I am not sure whether the update answers your question or not. Can you take a look and maybe clarify your followup question if it doesn't?
What does "cor(xu,xv,yu,yv)" mean?
Also, you might be interested in the paper [Using cross-validation to evaluate predictive accuracy of survival risk classifiers based on high-dimensional data](http://dx.doi.org/10.1093/bib/bbr001). [Gene Expression–Based Prognostic Signatures in Lung Cancer: Ready for Clinical Use?](http://jnci.oxfordjournals.org/content/102/7/464.short) would also be of interest to you. Both provide really wonderful discussions of this topic, although if I recall they don't provide full discussions of intra-tumor heterogeneity.
@NickCox, edit made. Thanks.
Not really homework :) Thank you so much for such a quick reply, it was very good explained. Thanks a lot! I forgot to multiply the probalities with the different ways..
If the distribution is exponential rather than Weibull, then the shape $\\sigma$ is constrained at 1.
The straightforward answer to this question is likely to involve the glmmPQL() function in the MASS package but there are lots of interesting issues involved - see http://glmm.wikidot.com/faq which is really worth a read.  You might get a very useful answer (here or Stack Overflow) if you'd like to focus on either the programming or statistical issues.  If both, then CV is the right place still.
@Mark: I'm no longer have access to academic resources, so I can't give you a bibliography, but the search term "Maximum Entropy Inference for Vector Quantization" should get you a fairly good starting point. Does that help?
The significance of the test statistic does not determine whether the residual deviance will increase for a smaller model (that always happens); further, please don't sign your posts; your name is given in the block on the right.
1) Are the integers counts?  2) you can have zero observed counts in cells with a chi-square. The issue is with low expected counts. 3) That you're getting 'exactly the same p-value for all your data sets' -- there's not enough detail here to tell what the issue is - what was this p-value and how were these simulations performed? It wouldn't happen to be that you used `simulate.p.value=TRUE` in `chisq.test` in R, with default `B` and got p= 0.0004998 would it? This is simply 1/(B+1). That's to be expected if there's a strong effect or large n
@AlecosPapadopoulos Thanks for your help.
The data is actually a set of genes involved in one process (Wnt signalling) v the set of all genes that are not part of that set. I've separated the two sets of data now but still not sure how to proceed.
@Kristina does the method described in the answer below for linearizing your models so summary can produce the values you want work for you?
In that case, you might want to try a variable "last month" which would be 1 if it was the last month of a quarter and 0 otherwise. Or you might want to use month as is and show that the idea in the documentation isn't true for your data, or only for one node. (3,6,9,10,12) is close to (last month).
You should almost certainly *not* have a categorical variable with 100 levels.
I can't comment on your answer because I don't have enough reputation... And now I still don't.
I can't comment b/c of my 1 rep so I had to type this as an answer. You might look into Jaccard Similarity. I think python scipy has implementations of it. [Jaccard...](http://www.code10.info/index.php?option=com_content&view=article&id=60:articlejaccard-similarity&catid=38:cat_coding_algorithms_data-similarity&Itemid=57)
"time series analysis" as a field of study. Basically I mean any thing that might trip up someone new to studying time series (of any type, and any type of analysis). I'm not looking for comprehensive answers. Check the question I referenced in my question to get a feel for what I'm trying to do here.
@BabakP I am sorry but that is completely useless to me, I do not understand it... (I have not learned how to use matrices to that extent in the field of econometrics). Thanks for trying though.
Have a look at standard time series econometrics textbook like Hamilton (1994) or Enders (2005).
This question appears to be off-topic because it is about how to do something in specific software. Please see advice in the Help Center about software-specific questions. Nor would it be a good fit for SO because requests for code or recommendations of code are off-topic there. How to calculate eigenvalues is surely one of the best documented problems in numerical analysis.
I'm not familiar with Bishop's notation, but IIRC, the loading $\\lambda_{ij}$ in matrix $W$ equals the correlation of observed variable $i$ with latent factor $j$ only in the orthogonal-factor model. In a model with correlated factors, matrix $W \\cdot C_f$ contains the variable-factor correlations if $C_f$ is the correlation matrix of the factors (not equal to the identity matrix).
You can certainly turn state means into a $z$-score (or something loosely analogous, in your suggestion), but this does not address the additional information in each state's SD (assuming there is). If there is independent info in the state SDs, that will need to be represented somehow in addition.
See my answer regarding Bayesian rating systems below. Applied rating contexts typically have hundreds or thousands of objects being rated, and the aim is often to form the best estimate of the rating for the object given the available information. This is very different to a simple two group comparison as you might find say in an medical experiment with two groups.
@Daniel: You need some way to get an expression for $P(E_1, E_2 | H)$, or things convertible to it such as $P(E_1|E_2, H)$.  The easiest fudge is conditional independence given the results of the gold standard: assume $P(E_1, E_2 | H) = P(E_1 | H) P(E_2 | H)$.  This can be a bad assumption, and can report a more certain probability (closer to 1 or 0) than the "true" probability.  To be conservative, you could try to find the "least informative" joint distribution $P(H, E_1, E_2)$ that's compatible with what you know (i.e. has the same marginals $P(E_1, H)$ and  $P(E_2, H)$).
thank a lot for the paper. Regards, Stefano.
Thanks, B. I'll take a look at it.
Thanks for the link to a possible duplicate.  The questions are related but they are not duplicates: here, you are asking for how to analyze the data, not for a sample size.
On the contrary, I think most people would use 0, 1 here.
+1 This adds thoughtful and useful analysis to the other answers.
I did not test the normality of the response, but the normality of the residuals.
The value '2' is a rough t critical value; I'm inclined to agree, I'd tend to use either an actual $t$ critical value or 1.96. At first glance, the first variance estimates seem to relate to the contribution of the fixed effect parameter uncertainty to the variance of the predicted value and the second looks to be sum of that and the contribution the random effect variance.
I am actually using the cross-validation to tune the parameters.  Then I have a separate TESTING data. My question is basically wether having a 100% in the n-fold stratified cross-validation of my TRAINING data is a bad symptoms  (like overfitting the training data  data). Thanks :)
I am not sure to understand precisely what you are after or what “significant” means in this context but you might be interested in measures of “inter-rater agreement”.
This seems to be a standard textbook question. Please read the [self-study](http://stats.stackexchange.com/tags/self-study/info) tag wiki info, add the tag and then let us know what you already understand, what you've tried, and so on.
Not quite. Lets say there is another cell with its prior being set to $r$, then its posterior is $r\\frac{1}{1 - pq}$. How is this arrived at? I understand why it would change and go up, but don't follow how it is calculated to that expression
What package is `c_hat` found in?
Does Var(residual i) calculated from the formula Var(error)*(1 - hatvalue(i)) have any meaning?
Presumably we are to imagine observations on several birds and on each of four characters on each of these subspecies. As always, the idea that a test is the Holy Grail to seek in an analysis is not obvious: you are likely to get much more out of (say) a correspondence analysis. Whether your three grades are ordinal or nominal is not quite clear. Whether you can combine characters is not predictable by us in advance, but why go to the trouble of collecting detailed data only to want to mush it altogether?
Please, work on improving this quick reply.
+1 for a useful idea that is easily executed and theoretically valid. One small correction, though: there will be $3n$ values observed over $n$ time periods. An unbiased estimate of the variance will divide the sum of all $3n$ squared deviations by $2n$, rather than $n-1$ or $3n-1$ (either one is substantially different and is wrong). One probably should plot the individual variance estimates (one per time period) over time, too, to evaluate whether it's legitimate to pool them all like this.
the correlation is calculated for each pair of variables separately. So conceptually there is no difference, you just need to repeat the analysis; in R you could use lm(y~x)
+! @whuber you go to a lot of pains to make really nice simulationa and graphic illustrations.  If the OP had trouble understanding parts of my answer and Peter's This certainly shoyuld make things clear.  I think my discussion with the OP which you may have seen all or most of, indicated that when he adjusted the pooled data the the ratios were very similar. The other issue is that he compared at one specific point on the regression curve and it was not clear that all the curves used truly comparble points.
Normally one uses $t$ for time, which would be the "x-variable," so I'm wondering about the connection between your question and the equation you have provided.  Where in that equation is the one month return?
Excellent! This is exactly what I was looking for, thanks!
Of possible interest: [Is there any statistical test that is parametric and non-parametric?](http://stats.stackexchange.com/q/18450/930) (At least, many of the comments and points raised in answering this thread suggest that your first question is a duplicate.)
Do you really plan to look at over 22,000 of these plots after you have waited for them all to be computed? Instead of creating such detailed images, you would seem to have a pressing need to screen the data first for "interesting" pairs of variables and to display their covariation in an extremely succinct and simplified fashion.
R does have a package called dlmmodeer that does UCM. However, it is not as intuituve as other software packages that I mentioned. Check these 2 webiste: (1) http://cran.r-project.org/web/packages/dlmodeler/dlmodeler.pdf (2) http://www.jstatsoft.org/v41/i04/paper
That's really helpful @Andy. Thanks for the detailed answer.
It is noteworthy that the edit almost completely changes the question.  (This is no longer a comparison of a distribution against a reference: it involves multiple comparisons and *no* reference.)  There are now good reasons to suspect correlations among the counts, too, which could be induced by the nature of the clustering and the nature of the processes themselves.  Answers can be derived by ignoring these complications, but they might then lead you to erroneous conclusions.  A good answer would incorporate some model (or understanding) of the relationships between processes and clusters.
@netuser01 you can accept my anwer, by clicking on the button to the left of my post (the hook).
What kind of data is this ? We often find chronological/time series data having "unusual pdf's" and after suitable filtering/modelling the resultant series has a much more manageable density. Please describe your original data and perhaps post it to the web for guys like me to ponder over.
ok, thanks, it's pedantic, but correct. I made an edit.
"Covariance estimate" is a term from the output of SAS, I think (this is the estimate of an entry of the random effects covariance matrix).
You mean like |---------------:---------------| ?
What did you try
Such a response format is typically called a Likert item (or a Likert scale but that's somewhat confusing because the whole point of Rensis Likert's work was to combine several such items to form a single scale). If you use that term to search for more info, you will see that there is a lot of material on the web and several questions on this site addressing various aspects of this type of data.
+1 @ John's reply. You need to have a factor variable for `X2`. You are practically saying to the model to have a different intercept for every `X2` value but if you have $N$ different values in `X2`( $N$ being the # of data points in your sample) your problem is overdetermined. If `X2` has some meaning (eg. BMI) and you can derive *meaningful* categorical values based on it, you are fine. (Don't chop stuff arbitrarily that nonsensical.)
I'm not sure I can easily translate these into a solution to my problem, but they are both fascinating reads. +1.
In other words, you want to know if Simpson's paradox can appear with mean differences instead of odds ratios?
@JeremyMiles Thank you.  I fixed the issue.
One conventional approach to this problem, which has been around for at least a quarter century (it was in Primavera in the late '80s), is to specify [Beta distributions](http://en.wikipedia.org/wiki/Beta_distribution) for task durations in the natural but somewhat indirect way of giving an "expected" (modal) duration, a minimum, a maximum, and (perhaps) additional information. (Without additional information, many people assume triangular distributions.) You should be able to read about this in the project management literature, such as http://www.routledge.com/books/details/9781420083170/.
More generally, statistics with a Chi square distribution of any sort are always positive so their sign can't give any information.
the data is pretty skewed! With sqrt() transformation I can also normalize but nevertheless both groups never pass D'Agostino omnibus normality test.
However my biggest problem is that one test pass from not significant to significant if i use t-test instead of mann-whitney. I thought that for high numbers the two tests would give comparable results!
Of possible interest: [Nonparametric equivalent of ANCOVA for continuous dependent variables](http://stats.stackexchange.com/q/41270/930).
The central limit theorem applies to the Bernoulli (indeed, that's how the Gaussian distribution was first discovered ... by De Moivre in the mid-1700s, as a large sample approximation to the binomial); but take care - the CLT describes what happens in the limit, to a standardized mean (or a standardized sum). As whuber suggests, there's a lot of material on it here.