tttjjj/elt_bench huggingface dataset:
@Simone For the fitting, the category $\\ge 5$ was replaced by *all* values $5, 6, 7, \\ldots$.  In this fashion we avoid making an arbitrary assignment of a single value to this category.  The price we pay is to push the arbitrariness back to our original assumption of a Poisson distribution: there's no avoiding making some assumption, if meaningful statistics like a mean or SD are to be computed.  But at least this assumption is clearly stated and it serves as a useful, interesting point of reference for evaluating the entire distribution.
Hi, Jim. Welcome to the site! Is this, by chance, a homework exercise? If so, we should mark it with the [tag:homework] tag. They receive somewhat special treatment on this site.
It does integrate to 1 for every value of $\\alpha$.
Thank you very much, went beyond the call of duty, very helpful.
Is "sigma" the *standard deviation of the data* or is it perhaps the *standard error of estimate of the mean*?  The distinction is crucial!
Thanks @onestop for your nice answer. Actually $\\mathbf{A}$ is kind of a relationship matrix and I want to use it in model fitting and interested in its value after model fitting. Any suggestion. Thanks
Also, while we cannot give any conditions on $F$, my above argument shows that we can very well do so for $F'$. In particular, the "value adjusted density" ($\\alpha F'(\\alpha)$) has to be sufficiently large relative to the parameters $N$ and $\\delta$. I wonder whether much more could be said if I imposed some regularity conditions on $F'$, like single-peakedness.
[flow map](http://en.wikipedia.org/wiki/Flow_map)
Making a clear distinction between *data* following a normal distribution--which they never do--and the underlying population or process following a normal distribution--which is a matter of what assumptions one makes--would greatly improve this answer.
@Jens The problem is that there are many more integers (just to illustrate the point) between 10^12 and 10^15 than there are between 0 and 10^12. In fact, there are about 999 times as many. So if you sampled each one with the same frequency (as you would with a uniform random distribution), then your results are exactly what I would expect.
For someone who supposedly isn't very good at English, you seem to fake it rather well. I'm always impressed by people who can discuss topics in more than one language. I can barely work in English, but my wife is Dutch--she can speak 10 languages.
I don't know unscented KFs much, but the MCKF sounds a lot like the EnKF. You might want to check that out, and possible comparisons with the unscented KF.
@Scortchi, I was unable to get past these terminology differences, frankly. I also know that normal approximations are close to being useless in the tails, so the 6 sigma probability $10^{-9}$ may be off by a factor of 100 or 1000.
Welcome to the site, @jakub. CV has several goals, 1 of which is to develop a permanent repository of actionable info, & 1 concern re that is linkrot. In that light, can you give an overview of the info at the linked site? Since you're new here, you may also want to read our [FAQ](http://stats.stackexchange.com/faq), which discusses these issues among other aspects of CV.
Firstly, thanks for referring to my note. I think the first answer is correct. I'll try to improve the note.
Good answers to this question will need to know why would you want to smooth out these values.
@mbq: Thanks for migrating the question. Also thank you for hinting at the other questions.
Did you have any success in finding an implementation?
Also, there really is no single "best" test for reliability. Most inter-rater reliability statistics I've encountered are for nominal, ordinal, interval or ratio level data. It's possible that with continuous data, something more like correlation or rank correlation statistics may be more useful. I'd encourage you to look into what sorts of techniques are common in your particular field and/or to say more about the scale involved if you'd like a more informative answer.
@Flask - I don't quite understand what you mean. Are you saying that the regression model (i.e. a sigmoid) is an estimate itself? My point is that you should pick a model up front that makes sense. I would use AIC to justify using a sigmoid. Then, having established your model, you can perform statistical tests to compare their estimated parameters, which is directly analogous to comparing the estimated slopes of two different pairwise linear regressions. This approach seems simpler, but perhaps less flexible(?) than Oleg's answer. Let me know if I'm misunderstanding something.
An alternative is to do an exact version of the chi-square test (that is, use a chi-square test statistic, but use its exact permutation distribution conditional on the margins).
You can probably take another tack and say it's definitely pointless to calculate significance.... but yeah, that's an unlikely outcome to have occurred if there really were no difference in speed... which is what significance means in this case.  It's an unwieldy way to be discussing things that you can avoid if you just avoid the statistic.  Think of things like a t-test as a crutch for when you don't have better options.  You have lots of better options.
the usual answer to non-linearity in context of ARMA models is ARCH/GARCH models. For your first question, it is probably possible to construct PACF using rank correlation concepts, it will boil down probably to independent component analysis. As for the second the answer is probably no, but for this [article](http://arxiv.org/pdf/0712.0679) might be of interest, since it involves non-linear specification and ARMA is a subclass of it.
@ta.ft: two approaches are a) incorporate as much external knowledge about your application and data into the modeling in order to drastically reduce the number of models that need to be compared (= decide hyperparameters instead of optimizing). It may be overall better to change to a classifier that has intrinsically meaningful hyperparameters, i.e. where you can know from application and data type what the hyperparameter should (approximately) be. b) compare the few remaining models by proper scoring rule. E.g. Briers score has far better variance properties for many classifiers.
Ah, sorry for my ignorance.
@Peter, OK, I follow your idea, I have changed the question accordingly.
Could you please clarify why the intercept must be included so that the mean of the residuals is 0? I noticed $y_i -  x_i'\\beta = \\epsilon_i$, so $E(y_i -  x_i'\\beta) = E(\\epsilon_i) = 0$, since each error term having a mean of 0 is another assumption of OLS. So, is the error term, $\\epsilon_i$ the intercept you are referring to?
Can anybody atleast tell that why is this off topic ? Its REALLY very rude to close this off without suggesting the appropriate place for this.
You can choose your units of measurement--meters, yards, parsecs, whatever--without materially changing the problem.  So, choose a unit in which the rod has length 1.  Done! :-)
Also see [here](http://stats.stackexchange.com/questions/68834/what-is-the-benefit-of-breaking-up-a-continuous-predictor-variable/68839).
Hi Anne, could you elaborate on your answer to contextualise it in OP's situation? I too am interested in how you would implement the problem in a CFA framework.
I tried, spectral analysis, but there is only one library in R for that which doesn't look promising. There are a couple of other clustering method, which usually spit out the label for each node; but they don't sort and re-order the matrix. It is also challenging after having labels, to reorder the similarity matrix.
**More hints**: (1) How are $\\overline{X}$ and $S$ related to the *width* of the CI?  (Write a formula.)  (2) How is this width distributed?  (3) Can you find the expectation of this distribution?  (Be careful: remember that the expectation of a square root is not usually the same as the square root of an expectation.)  If you're still stuck, let us know which of (1), (2), (3) you have figured out.
So if I understand you right I should only use one training and one test set for all models? Can I still use my 5 variables for the last model then or is it a risk for over-parameterize? And isn't it a risk with just one training and test set - since it will be very much depending on where the split will be with the relative little data I have - or is that nothing to worry about? Otherwise it does feel like this would be the most correct way of doing it.
Have you tried logistic regression with a LASSO penalty? That's the first thing I'd try.
CCA looks useful to some extent. What I really want to do is use A to predict B. But I don't want to predict individual variables in B. I believe there is complex structure in B, so I want first to identify that structure in B and then see how well I can predict that structure with A. But at the same time, A has complex structure, so I also want to identify structure that exists in A and use that to predict B. Not sure if I'm using valid terminology here, but hopefully it gets my point across.
One other thing:

I did find this link which talks about t-tests for overlapping samples. But I have no idea how credible (if at all) it is.

http://nces.ed.gov/nationsreportcard/tdw/print_page.aspx

Is it of any value?

Thanks
what is kpi? key performance indicator?
how are your test/training set constructed? For example, are they random partition of the same dataset or are they coming from two temporally disjoint datasets (e.g. the test set is composed of observations samples before a certain date and the train set of observations after that date)?
I don't see why I need it in this case. I'm supposed to treat the time intervals like they are conditionally independent given the observation. Notice that the product of the joint density is just from $i=2..11$. I don't think I'm supposed to get a sequentially updated formula here, just a single formula for the posterior $p((\\mu,\\alpha)^{t}\\quad |Y)$.
you need weights to get comparable price-levels results, usually volume of goods consumed - implicit equal weights for all goods are evidently wrong, since some of the goods have no price => they might have 0 quantity
By sampling a small fraction without replacement and limiting the tree depth, greater diversity is introduced so i think some form of regularization is warranted.
+1 for comprehensive and comprehensible answer (with code!)
`weights` is what you want; `w` is a vector of starting values for the IRLS weights.  The ability to use the latter helps sometimes especially with the bisquare psi-function, which doesn't have a unique minimum and can wander off into fail-land.  I personally like the bisquare, as it can completely remove extreme outliers.  My thought process goes like this:  Are my errors likely to have tails more like a t distribution or an exponential distribution?  If the former, bisquare; the latter, Huber.  (The MLE for a t dist'n location is redescending, like the bisquare.)
(+1) Welcome to the site. That is quite an exhaustive list! :)
If your question is to retrieve the values of the density, then you might try something like: `density(x)$y` and `density(x)$x`.
Of possible interest: [VSS criterion for the number of factors](http://stats.stackexchange.com/q/32669/930).
The OP's aov call has three random effects; to replicate that with `lmer` my above code is correct.  Your `lmer` code only has one random effect.  Which is correct will depend on the context.
It's not clear from your description what the study designs are (e.g. RCTs, controlled clinical trials, single arm observational trials, etc.). That's going to be the key to determining how to meta-analyze them and whether or not you will need advanced techniques like meta-regression or not.
@poorly_built_human My example was not very good, I tried to use animals in both cases but things like cold-blooded vs. warm-blooded or furred vs. feathered can be treated as separate multi-class classification problems. The canonical multi-label classification problem would be identifying topics in a text.
If all you have are two variables, pre and post, and some values are missing, there is almost nothing you can (seriously) do except omit observations with missing values, which SPSS will do any way. Naturally, you could play with home-grown imputation, such as using the mean of non-missing values to impute missings, but that would in practice be unlikely to help, unless you have strong independent grounds for any scenario.  If you have other data, then there might be other things to do, but you have to tell us more about what else you have. Absence of add-on doesn't rule out downloading R.
You might be interested in this related thread: [Visualizing Likert Item Response Data](http://stats.stackexchange.com/q/3921/930), which looks like a duplicate.
+1 again, if I could, for including the testing!
Is this useful? http://stats.stackexchange.com/questions/6067/does-an-unbalanced-sample-matter-when-doing-logistic-regression/6072#6072
@Amanda I don't understand your point :)
Nice, how much did you gain by using Gibbs in terms of rejection rate?
Well, note that the model fit with orthogonal polynomials will have the exact same fit (i.e. the same $R^2$, same fitted values, etc.) as the model fit with the raw polynomial terms. So, if you're looking to relate this back to the original data, you can look at the coefficients for the raw terms but use the orthogonal polynomials to do inference for the individual terms in a way that "accounts for" the dependency between them.
Was his name Ronald Coase? http://en.wikipedia.org/wiki/Ronald_Coase#Quotes
I can't see that your rephrase addresses the key points. You can collapse your profiles to total execution times to get two values to compare. But if you wish to compare profiles component by component that's a multivariate problem. Also, if you think that data should be summarized by means and standard deviations Wilcoxon isn't suitable at all, but you should be addressing whether a t test applies.
I had to give a shout out to the textbook used for my Bayesian class. We didn't use Gelman, but I follow his work on my own. Hell, while we're spreading the Gelman love, here's his blog in case anyone hasn't seen it. He posts quite regularly and the articles are generally pretty interesting: http://andrewgelman.com/
MAE, RMSE, cross-validated MAE
Thanks for your reply. As i am working with GPS data, whenever the GPS does not detect the location of vehicle, my depandant and all independant variables become zero, because the independant and dependant variables calculated based on GPS information.
Could you elaborate a little bit more on the "different categories of the classification"? I'm thinking a multi-level model might be useful here...
@COOLSerdash You're absolutely right. But I think there is a more benign interpretation possible: the professor recognizes that high variability might be a marker for lakes that do not belong in the study in the first place. The crux of the matter, then, is to establish objective criteria to identify what the study population is and then compare the observations against those criteria.
Thank you for the detail! I do not understand why you would like to calculate the posterior probabilities without the Foward-Backward (FB) algorithm, because the FB is the algorithm to get the posterior probabilities.
I would start by looking at logistic regression, (a generalized linear model). It doesn't matter that much how your continuous variables are distributed. If anything, it's more a question of whether they predict the outcome in a linear way (vs whether a transformation (like log(varB) has a more linear relationship). It would help to know the no. observations involved. I would also start by examining crosstabulations of the outcome vs the non-continuous predictors.
$2^{20}$ takes $6$ minutes :-).  (That's about the limit of my patience: by that time, we should be content to sample from the permutation distribution.)
It will be difficult to de-bias the severely biased estimates that result from trusting results learned from a properly penalized procedure and taking them to an unpenalized setting.  It is best to use a single procedure, or to use pre-conditioning (model approximation) to represent the penalized model more simply.  If you do that, then quadratic penalization probably works better than L1.
Unfortunately I do not know of one. Here is a possibility. I see in the FAdist R package there is a reference given under the gamma3 function documentation. It is: Bobee, B. and F. Ashkar, 1991, The Gamma Family and Derived Distributions Applied in Hydrology, Water Resources Publications, Littleton, CO.
I must not being understanding something.  You specified in the problem that $\\sigma_{Y_2}=\\sigma_{Y_1}$.  So the relationship is just that they're the same?  If this isn't the case then please provide some more detail as to why not.
This is also an example of testing the "wrong" null hypothesis.  Its not the conclusion you're interested in.  A better hypothesis test would be that the weight loss is less than 5kg vs greater than 5kg.
You say this sequence 'may represent the correlation between a couple of measurements'. Do you have the data for the measurements? If there are only a couple of measurements, how can you derive a sequence of *N* correlations from them?
@gung: I think you're right in your interpretation - changed the text, better? I haven't used the regression models myself although I've encountered them recently in a course. Tt's an interesting alternative to regular Cox-models that I've used a lot. Although I probably need to spend more time digesting the idea I feel that it's probably easier for me to explain to my patients since I frequently use KM curves when explaining to my patients. HR demands that you really understand the difference between relative and absolute risks - a concept that can take some time to explain...
In the link I provide for Pollard's book, look for the .pdf "Classical" or "old-Classical" (but Pollard's style and notation are difficult). As for the second issue, what happens when we have a sample of size $n=1$?
It worked! You are a savior! Thanks a lot :)
There is a very similar question in http://stackoverflow.com/questions/15111408/how-does-sklearn-svm-svcs-function-predict-proba-work-internally which is directly related to a certain implementation of SVM (Python SciKit using libSVM), but I think it will help you.

See the documentation/examples in http://scikit-learn.org/dev/modules/generated/sklearn.svm.SVC.html for more details.
Thanks, it's certainly possible using the variational lower bound. I've added another solution using Monte Carlo integration.
I think this has been discussed before in the setting of AUC. In this analogue F-measure=Youden Index.
AFAIK SVM have the same problem of random forest. They do not predict well outside of space where they have been trained. Probably neural network would be a better solutions
@gung Although the rule of thumb deals with expected frequencies, the problem of sparse cells is also identified by the actual sparseness of the cells and so having raw numbers below 5 is a concern.
Related to: [SVM with only one type of label](http://stats.stackexchange.com/q/41324/930).
You *could* be right, but there's a lot in the question suggesting the response is binomial.
What do you means by "proportionally"? Proportional to what?
... I'm kind of wondering whether anyone should report p-values without any understanding of the underlying theory ... Anyway, the test I've made is a baseline Z-test to compare proportions. But to obtain the said proportions you should still convert your ratios in proportion (first formula: proportion = number of intronic/(total number). What program do you use?
The variance decomposition works for any model fitted by OLS, of course only for the transformed variables. As for averages, non-linear transformations and taking variances can't be reversed.
It's here: http://onlinelibrary.wiley.com/doi/10.1111/1467-985X.00266/abstract
Context would be helpful: I suspect it's the model that's empirical rather than some special regression technique's being used. An empirical model is one flexibly parametrized to cope well enough with the unknown form of the true relationship between predictors & response (e.g. using polynomials or splines to allow for non-linearity); a theoretical model is one whose form & parameters are determined by theory (e.g. estimating $V_\\mathrm{max}$ & $K_\\mathrm{M}$ for the Michaelis-Menten kinetic model from substrate concentration & initial rates).
+1 for linking to the full essay rather than one of the all-too-common oversimplifications - it's not long and is still relevant today and well worth reading in full. Hill's criteria for causation are about much more than just pyramid / piling up of evidence though.
The following CV questions also discuss this material: [Difference between generalized linear models & generalized linear mixed models in SPSS](http://stats.stackexchange.com/questions/32419/); [When to use generalized estimating equations vs. mixed effects models?](http://stats.stackexchange.com/questions/16390/).
The way I've seen it, most text books define "random variable" to be a mapping from a sample space to the real numbers. Essentially, a "random variable" is real-valued.
Could you cite the article, or at least outline the methodology it proposes?  It's hard to come up with a "gentler" way of doing this if we don't know the baseline :)
re: macro, agreed. overall, I'm really grateful for the support from you guys!
(+1) Thanks for a good advice. Your experiences are valuable to me. For some of my questions I never got a satisfactory answer (well, so far). But for some others I got amazing complete answers. This means that we can contribute to the community by questioning but finally the community decides how to answer to our questions. Therefore grateful to your nice advice the best is to get benefit as much as we can from the given answers and kindly efforts. Behind anything my appreciation goes to the time being spent viewing, answering and commenting my posts.
@xpda That's a strange interpretation, because everybody either is, or is not, descended from any particular individual, as can be established through DNA testing.  I think the way many people may have been understanding your question is if we pick an arbitrary person "$A$" in the year 1300 and we select a *random* person alive today, what is the chance they are descended from $A$?  This is answered by estimating the proportion of today's population that is descended from $A$.  We could also take $A$ to be randomly selected.
The residuals cannot all have the same variance, yes, but the $\\varepsilon_i$ are the errors, which are assumed to be iid. The residuals are the observed values minus the fitted value, which is different from the errors, the observed values minus the unobservable true means of the $Y$ at the given value of $X$.
oops sorry for the late answer - my connection was lost for a while.
A ROC curve ( http://en.wikipedia.org/wiki/Receiver_operating_characteristic ) and a F1 scores ( http://en.wikipedia.org/wiki/F1_score ) are probably good measures for a *first* performance comparison. Is there a particular performance measure you need to implement and you want specific information?
$t(x_i a)$ in the loss function above is an approximation derived from the model $f(x,a)$? The model would be something like regression, right? I think I am misunderstanding something, "approximate a function using a model by folowing the gradient of the loss function", but it looks like the loss function is using the approximated function, or are they different functions?
@overrider Under which conditions that transformation would be a valid 'node' in the model? How should the MCMC algorithm treat it? In PyMc, it is just a method that returns the results of a transformation, but I can't see how PyMc's support for deterministic nodes is related to MCMC algorithm at the abstract, mathematical level. There must be a name for this software feature in the MCMC literature, but I can neither find that name nor resources that discuss its aspects.
You should accept the answer.