tttjjj/elt_bench huggingface dataset:
(+1) In rounding $1041/3742$ to $0.27$ you destroy all but the most significant digit in your probability(!), which should equal $1.116\\times 10^{-10}$. It may also be worth noting that we should interpret your "If we regard" as an *approximation* to the true probability indicated by your model, which is of sampling *without* replacement. The sample consists of the 44 initial words out of the population of 3742 words. This lowers the answer to $8.41\\times 10^{-11}$ as computed by `phyper(32, 1041, 3742-1041, 44, lower.tail=FALSE)`.
If you are going to use multilevel modeling, or latent growth modeling (from structural equation modeling framework) which has similar capacity in handling the type of data you have, it would be very helpful to first read and study these methods.  Most books on these use other softwares such as Mplus, R, HLM, MLwiN, and it's easier to find examples outside SPSS environment (which means making life easier in a long-term).  Also these methods are often quite demanding on sample size, which is not always easy to get with intervention studies.
@Dale, I'm still a little vague about your use of "nesting".  It sounds like variable (1) is effectively the interaction between (2) and (3).  In any case, AIC/AICc are extremely general, and it's hard for me to imagine why they would not be appropriate here. If you could communicate some intuition about why the "nesting" makes you suspicious, that could be helpful.
@jb Presumably $T$ is *huge,* otherwise the OP wouldn't be asking to avoid the matrix decomposition (which is implicit in `rmvnorm`).
What do you mean by "ANOVA table"? The example you show isn't what I would normally call an ANOVA table.
@Andre Holzner - what you say it true, but, but, but... doing this random forrest is actually a form of regularisation, in a similar class to ridging.  I'll tell you a secret, a regression tree is actually a linear model - similar class to splines.  putting my Bayesian hat on, the random forrest regulariser would likely correspond roughly to the "spike and slab" priors used in Bayesian context.
@Fr. I added a PS (well, it's so ugly code that I have no doubt someone will come with a better one :-)
A quick note related to your question about coverage probability: We have some unknown quantity that we cannot observe, in this case, the mean. We can construct a confidence interval, which gives us a rough idea of where the true value of the parameter is. Depending on what points we sample from in our population, we will get different confidence intervals. Coverage probability is the proportion of those intervals that contain the true value of the parameter. See Wikipedia: http://en.wikipedia.org/wiki/Coverage_probability
No, then all bets are off.  It is best for those models to do joint (composite; chunk) tests grouping all the parameters that go with each predictor.  The degrees of freedom in such tests is equal to the number of unconstrained slopes, which gives a perfect multiplicity adjustment.  And if you estimate cutoff-specific effects of variables or make graphical judgments doing same, you'll need to estimate the effective degrees of freedom to do the multiplicity adjustment.
I can't see that there IS a general solution, at least within logistic regression. With chips, I think survival analysis is often used, but even then, with no failures, I don't see what can be done.
+1 Welcome to our site, Sam! Thank you for offering such an informative reply.
(+1) for offering a very sensible criterion/content for "highly non-linear" (that linear approximation may make matters worse).
Similar question: http://stats.stackexchange.com/questions/10613/why-are-p-values-uniformly-distributed-under-the-null-hypothesis
+1 FWIW, I recently posted a worked example along the same lines at http://stats.stackexchange.com/a/34894. Although it would be misleading to construe it as "plain English"--it does not shy from using technical terms--I made an effort to explain as clearly and briefly as possible what is going on, what assumptions are made, and how one works with a parameterized family of distributions to produce an estimate based on data. For some, this might be an informative adjunct to your answer here.
Are you saying that you are going to start with a set of known cases, next perform your biomarker test (collect data), and estimate the sensitivity?  And you will start with a set of known controls, collect data, and estimate specificity?
Yes, divide roughly equally to get an idea of the performance as a function of alpha, then near the maximum you can use a finer grid.
This question might be a better fit over on math.stackexchange.com .
Can you define "MRF" & "CRF"?
Can you be more specific? It is a topic for a book.
Or 97 df if the model includes an intercept as you often would when there is no differencing.
@whuber: It may appear not to use the information, but if you investigate further you should find that the number of unique items is a sufficient statistic. For example, if you take a sample with replacement of 4 items from a population of $n$, the probability of getting 3 of one item and 1 of another is $\\frac{4}{3}$ that of getting 2 each of two items, no matter what $n$ is, so knowing the detailed frequencies gives no more useful information about the population than simply knowing there were two unique items found in the sample.
Related http://stats.stackexchange.com/q/22329/3277 ?
I agree that good scales should have alphas higher than .8 or .9. I was just surprised that a two-item scale would have such a high alpha. I knew that it was possible to get such a high alpha, but I was wondering what was the more plausible scenario: (a) the authors made an error, perhaps, dare I say, inspired by the need to meet the ".80 threshold of publishability"; or (b) the two items actually were sufficiently highly correlated to result in such an alpha. I think the data you show looks plausible given two very similar items, and therefore I'm inclined to trust the reported results.
+1 Thanks for the detailed answer. There certainly are complicated ways in which the floor function affects the variance.
I believe that "but $X$ and $Z$ are independent" is the statement the O.P. is asking about.  The answer by Paul Staab directly addresses that issue in full generality.
For technical part (the tools) I would recommend two books as good staring point both fro O'Reiley: Collective Intelligence (with Python code), Machine Learning ( with R code ) ...tey capture topics related with yours. Next step might be Manning website...
For methodological part I would recommend Semantic Web group on LinkedIn.
Including a time-varying coefficient in the model is also a mean to check the PH assumption
+ for the hanging rootogram!
@Srikant Just a few words about R code (per your request): you can use `replicate()` whenever you want to call a given function like `rbinom()` $k$ independent times; the `{*}apply()` family functions are very useful to work with matrix, data.frame or list and make a call to the same function by column or row; it is recommended to avoid the underscore for variable naming (and use a dot instead), but it is subject to debate. Otherwise, I also like simple and explicite R code like yours because it will be understandable to everybody. I gave my +1.
This may be helpful: [Bayesian models for categorical data By P. Congdon](http://books.google.com/books?id=rzTegn21i58C&lpg=PA167&dq=congdon%20zero-inflated%20model&pg=PA167#v=onepage&q&f=false)
That formula is reminiscent of the formula for the arithmetic mean of a lognormal distribution, which might be appropriate for strictly positive values with standard deviation large compared to the mean. I don't know how to format a formula either, but you can see http://en.wikipedia.org/wiki/Lognormal_distribution#Arithmetic_moments
I think you need to give a bit more information about your model. What are your data (are they individuals who have may or may not have been admitted)? What is your precise model (you said that you coded it as binary, but then used multinomial logistic regression: which is it)?
Hmm, I ran the code with your modification to get this: `[n/float(sum(posterior_A)) for n in posterior_A] = [0.060, 0.12, 0.05, 0.198, 0.312, 0.079, 0.178]`. The issue with the last element still persists (it is supposed to be approx. `0.3`).
You can do the computational cost per iteration, though. Of course, for a given problem there might be different ways of implementing the same EM procedure with different computational complexities.
Exactly what does the error message say?  (I feel obliged to add, though, that automatic--"mindless"--procedures such as stepwise regression and "pressing buttons" are a sure way to produce indefensible results. You should take @Frank Harrell's comment to heart.  [Search our site](http://stats.stackexchange.com/search?q=%2Bstepwise+%2Bregression) for more information about stepwise regression and reasonable alternatives.)
Sorry, I still don't understand what is the problem, so I try to go into more detail. My suggestion was that it doesn't matter what actual form the Z has. As long as you can calculate p*(x) fast, you can integrate p*(x) numerically with any precision you desire. Z is equal to that integral.
If one wanted to test whether an effect is different between more than two groups, would an ANOVA comparing the model $y_i = \\alpha + \\beta x_i + \\gamma g_i + \\varepsilon_i$ and the one shown in this answer, $y_i = \\alpha + \\beta x_i + \\gamma g_i + \\delta (x_i \\times g_i) + \\varepsilon_i$ be appropriate?
@DilipSarwate: In the second case, the mean and variance are not known. What is the LMMSE for $X_{n+1}$ then?
$I\\sigma$ where $I$ is an identity matrix and $\\sigma$ is a standard deviation is not right; it should be $I\\sigma^2$ since the diagonal entries in a covariance matrix are _variances_ not standard deviations. More to the point, $I\\sigma^2$ is the covariance matrix of three _uncorrelated_ random variables with identical variances $\\sigma^2$, and in this case, since you (seem to be) assuming joint normality, the random variables are independent as well. You "... can't see covariance like this..." because  that is **not** the covariance matrix of dependent jointly normal random variables.
The use of dummy variables doesn't change the method of analysis, really. ANOVA and ANCOVA and multiple regression are all the same model; in matrix algebra form $Y = XB + e$
@shabbychef What is a 'system'? Is the same as 'strategy'? If we had 6 systems and 4 strategies does that we mean that we have 24 columns? Is the goal to distinguish between systems/strategies or both? I am afraid it is confusing for an outsider like me.
ANOVA viewed through the lens of linear regression theory is, to me, the most appealing and unified method of learning and understanding the topic. A mathematician of most any stripe will feel quite comfortable thinking in terms of linear algebra and, in particular, projections. The $F$-test can then be understood both in terms of ratios of sums of squares and as a monotone transformation of a likelihood-ratio test, which give two complementary viewpoints. For one such text, Seber & Lee (2003), *Linear Regression Analysis*, 2nd edition, Wiley is a choice I can recommend, though it's not free.
How can they be independent if they are constrained to sum to 1?
For your first question, you can find the Wiki page of "Quasi-likelihood" from the "See also" column of Wiki page of "Quasi-maximum_likelihood". I think all the three Wiki pages need to be edited and expanded. As discussed in Page 139 of Diggle et al. (2002), "in the absence of a convenient likelihood function to work with, it is sensible to estimate $\\beta$ by solving a multivariate analogue of the quasi-score function (Wedderburn, 1974)."
Thanks for pointing out "contingency ratio" and "pointwise mutual information".
Nice reference! In the same vein, Categorical data analysis: Away from ANOVAs. (transformation or not) and towards logit mixed models. T. Florian Jaeger (JML 2008 59), http://j.mp/a8h763.
I guess I should have emphasized that I'm more interested in whether comparing the F-statistics p-value is a valid approach.
Thank you, cardinal.  As you probably know, the credit goes to *Mathematica.*  Here, the plot is produced by the command `RegionPlot[4 b^2 - 6 a > 0, {a, 0, 2}, {b, 0, 1}, AspectRatio -> 1/2]`
Sketch of proof of @Peter Flom's claim: any such family would be closed under arbitrary finite mixtures, whence so would their CFs, implying the moments would be linear functions of the parameters, leading to the conclusion that all CFs must be in the form $\\exp(-ikt)$, whence the *only* such family is generated by the improper uniform "distribution" on the real line.
I disagree with @StatMan 's advice to just use the same people who responded both times, because you are almost certain to get biased results: yours is a case for multiple imputation. Likewise treating the two groups as independent ignores the actual dependence existing in the before-after samples.
@whuber if you consider the direction of horizontal anisotropy, and if the ideal direction is 0°, then there would be three local minima at 0°, 180° and 360°. Although with this range, there would be a better defined minima around 180°.
I agree with you, i forgot about this question, i have already turned in my paper. But I do agree on your findings. I would love to see some work in a scale-space solution without the image resizing. Some late night i was considering to use different features instead of what is being used by Gall et al. I remember learning about "Basic Image Features" in a course. They describe some gradient information for each pixel accross scales. But feel free to come back here if you get some nice findings.
@Macro: How does it look?
Also consider the fact that someone might have to print it out on a greyscale printer. I've done this before - I used ggplot2 default colours (which look great on a screen) for an assignment, which I then printed out in black and white, and half of the colours couldn't be distinguished from the others! \\*blush\\*
@PeterFlom There are an infinite number of symmetric unimodal examples where s.d. is not a good measure of spread. An obvious example would be a t distribution with two degrees of freedom. The mean exists, but standard deviation is pretty useless -- the population s.d. doesn't even exist.
could you please introduce me some references related to undersampling for NER ?
In addition, many suggest a minimum of 20 participants per cell. Also note that the effect size estimates you get from published studies are likely to be biased upwards, as they do not take account of the file-drawer problem.
You are missing $\\pi_1 + \\pi_2 = 1$. This gives you the proper second equation.
@NRH, Thanks for catching (and correcting) my typos before I managed to see them!
I agree completely with what you're saying, but the reason for looking at tests here is to satisfy others. The situation is modelling possible extreme operational losses bassed on historical loss experience and the regulator needs to be convinced that the choice of distribution is supported by the data. What the regulator thinks is reasonable and what the business thinks is reasonable for the results can differ quite a bit! Using a (reasonably) standard statistical test may provide a somewhat independent approach to the justifying a particular choice.
Great. Thank you !
@FrankHarrell - I agree, but I would still class interval estimation as a kind of "decision theory" where the utility function is usually based on information content (i.e. conclusions which use more of the information we have are better) - and this is optimised by the posterior distribution itself, and possibly a posterior predictive if prediction is of interest.  Interval estimation provides a convenient summary of the posterior.  And good confidence intervals (e.g. based on MLE) provide a very good approximation to this when the information outside the data at hand is scarce
You mean x is D from the CDF you give? You can see the number of points Ne matters two.
This answer is correct. You may also be interested in [this answer](http://stats.stackexchange.com/questions/26147/why-is-the-sum-of-squared-differences-from-the-sample-mean-smaller-than-the-sum/26149#26149) to a previous question, which shows the same result.
NB: please note that by "left skewed" I explicitly meant that the *right* tail is short, not that the left tail is long. This is evident by how the points at the *right* of the plot fall *below* the reference line. Because the points at the left of the plot are (relatively) close to the reference line, it is incorrect to characterize this distribution as having a "longer left tail." The distinction is important here, because the right tail *ought* to have far greater influence on the estimated mean than the left tail does (whereas *both* tails influence its confidence interval).
"when I plotted depth-vs-temp curve and saw its variation over time, the fluctuation occurs at few places only."
@FrankHarrell Bad things do happen, yes. With the skewed distribution I drew at the top of my answer above (Gamma with shape 0.02), the distribution of the two-sample t statistic in medium-sized samples (say n=30) looks like the main cable on a suspension bridge - bimodal with two sharp peaks.
Almost never. Aside from sharing the word 'exponential' in their descriptions - which refers to *different* things in the two cases - do you have any specific reason to apply one to the other?
Thanks a lot for your response to my question! I already suspected that there wouldn't be one general way to proceed "correctly", I am glad that I know now your recommendation.
I found it on internet archive; I've changed the link to that.
@RobertSmith the proportion of instances that are inspected is directly related to the classifier's output because instances are ranked based on said output. Every proportion to be inspected corresponds to a certain threshold on the classifier's decision value.
I think this is just a question of terminology/jargon -- what you are describing is still a problem, and is due to overspecification, but I don't think we would refer to this as "overfitting" in a formal sense. I'll have to go away and read some bits on the distinctions to be clearer!
If plotting the distribution of artist counts helps to visualize the difference to a record collection of 13 artists which all have 3 albums, then, yes, this may be what I am looking for.
well, sort of answered. no substantial information was given in the answer to the simailar question regarding calibrating predictions
$H(M|C)$ is just the average over the distribution of $C$ of the entropy of $M$ under the conditional probability $P(M|C=x)$. There are many possible situations for which $H(M|C)=H(M)$.
Just to clarify: I wouldn't use OLS for a discrete, ordered dependent variable. It's fine in many other applications.
But that's just the rest of the Pythagorean Theorem!  Your first expression is the projection in the orthogonal direction, a space of dimension $n-(n-1) = 1$.
(+1) nice solution indeed, *however* the author still needs to store a bit more than just $\\hat{\\beta_1}$, but the idea is great, since allows fast updating with the smallest storage and requirements for the data. I personally would only go for this trick with the models that were at least statistically shown not to be inadequate.
I've made some other optimizations that get about 50% improvement. Updating the post now for more details - clever thought about the horner form, too!
I do not see how the probabilities play a role in your question. Could you edit your question to explain where they come from and what you plan to do with it?
Thanks! Could you also address the last sentence in my post if possible?
@gung Thank you for your comment. Can I ask you another question: even if I am not standardizing the feature vectors, in the aforementioned case, should I atleast scale the $X_2$ values in proportion to the total population of the towns before running the regression? Thanks.
You can do a Wald-Z test on a specific coefficient in a logistic regression, but this tests each coefficient individually and if you really want to see if there is a difference between groups then you need to test them together (as @lejohn pointed out).  Also the z-test can suffer from the Hauk-Donner effect sometimes where a very large coefficient is not seen as significant because the variance is overestimated.  Better is to use a full/reduced model liklihood ratio test.
Doesn't seem like you are comparing the rates of quitting as much as the admittance of not quitting.
No, there is no way to know a priori that a rbf kernel will perform better on feature set 1. And without actually tuning your C parameter in the linear kernel I would not conclude that feature set 1 is actually better.
@user4701 Sorry: IV -> "Independent Variable" or "factor" in your experiment.
I am not sure this is appropriate. From what I know, transfer entropy is tailored towards dynamic data. This is not the case here.
Can you provide a citation for the statement you read about SPSS's degrees of freedom being based on the listwise N? Also, if this is true, it will dampen, not exaggerate, significance as compared with results based on the total N.
You have the probability from the percentiles. What's the probability of getting over the 50th value - 0.5. What's the probability of getting over the 95th value, 0.05.
As far as I know, factor loadings (and hence scores) are only defined up to a multiplicative constant, so the signs are not meaningful
@what, yes that is correct.
sklearn has lasso: http://scikit-learn.org/0.13/modules/generated/sklearn.linear_model.Lasso.html
I am a little confused by the question. How many distinct models do you have? Three? In any event, there is no general answer to your point 2. HMMs can only output a sequence of existing states, so it all depends on which emissions would be closer to 'sliding' in your training sample.
What is the expected output for this example?
Quartiles here are with respect to the scores each respondent gets on 5 item, 5 statement scale The score should fall between 5 to 25 and this has been distributed into 4 quartiles. 5-10 is Q1 (first quartile) , 10-15 is Q2 , 15-20 is Q3 and 20-25 is Q4  . There are 5 types of needs and for each need, different number of  respondents fall in Q1, Q2, Q3, Q4. Now the statistically significant difference in no. of respondents in each quartile can be calculated with chi square, but if i want to compare the number of respondents who fall in Q4 for all 5 types of needs , then what stat can i use.
I don't agree with the argument: "if you want to publish plots on a a webpage" then "you should use a JS library". If you need a graph that is only available via R or SPSS, what are you going to do? If you want to dynamically generate graphs for a web page, then, if you can, using Javascript can be nice; but still, that isn't a sufficient reason.
Thank you. It is still unclear to me how to interpret these "insignificant" factors. Is the interpretation correct, that the model fits the available data well, and the two variables contribute to the model but not significantly at my chosen level?
Nice example but I think this is a little more complicated because it involves four predictors and not just one.
If the *condition* is vaguely defined, the issue then is not about the agreement between the tests, but what they are calibrated to detect. Poor eGFR and poor creatinine clearance are two indicators of CKD, both indicate related but not identical ways in which the kidneys are failing. Which test is "right" or "better" depends greatly upon the intended purpose.
I found a way to do this in PHP, finally: http://stackoverflow.com/q/13168568/105539