tttjjj/elt_bench huggingface dataset:
I agree with your misgivings about AUC. By accuracy, I mean the proportion classified correctly. I agree that it is unlikely that the poster is truly only interested in the binary variables, and I suspect that actually the amount of profit made plays some part. But if the discrete classification is really the only concern, then I don't see anything else to optimize but a classification measure. And if your classes are that strongly biased towards the profitable class then ignoring the data and always classifying as profitable will indeed be a hard baseline to beat.
One way to "rig" such a coin: Flip it once. Then just keep repeating the outcome you got on that flip *ad infinitum*. From this, it's simple to come up with many variations.
@probabilityislogic: We can make a slightly stronger statement: $n>m(k-1) \\Rightarrow p(n,m,k)=1$; also $n<m \\Rightarrow p(n,m,k)=0$. Yes the problem involves a multinomial distribution, but I fail to see how "at least one of the values is drawn at least $k$ times" translates into something computable as a CDF.
I just deleted my answer to reduce unnecessary noise. Thanks for your comment.
@Imorin I am using markov chain for prediction. I am clustering data to get the states of the markov chain process. Can increase or decrease of states can change the transition matrix's accuracy ? If  I have any wrong concept about it please correct me.
I'm afraid this answer in its current form may be less than helpful due to errors in it.  Skewness is a standard measure of *asymmetry*. It is not closely related to heaviness of tails: it is possible for the tails to be extremely heavy and the skewness to be zero (which is the case for any symmetric distribution, for instance). Please note, too, that it is impossible for $a_4$ to be negative, so the second half of this answer makes little sense. (Perhaps you confused kurtosis with *excess kurtosis*?)
A few simulations do suggest that the ratio will at least in some situations have a somewhat normalish appearance; it may be plausible for your data, but with some cases that seem plausible to me it appears to be substantially right skew.
Take a look at the option `V` of the `random` statement.
@Abe "marginal" is precisely "unconditional" :)
@GavinSimpson I mean the prior being derived from the other times series (external to the problem). It is time series about users, and from looking at all users I know e.g. that values should be between 10 and 30 (expressed as a normal distribution), and I want to use this information when predicting the time series of an individual user. This would be especially useful when predicting users with very short time series. In the extreme case, the time series of a user has zero length, and then the values should be drawn simply from the normal prior.
I'll wait and see if someone comes up with an idea about the history or the "why" and if not i'll accept this one
Thanks, yes I think you are right that confidence intervals would be more appropriate.  I will try the chi2 test.
And QLS stands for...?
Can you be more explicit? Are you looking for non linear relationship between two time series?
The first part of @Macro's answer [here](http://stats.stackexchange.com/a/29644/805) does the necessary derivation, and I think it's covered by the answers [here](http://stats.stackexchange.com/questions/33547/how-does-the-standard-error-work).
(Cont.) Since an item in your example can belong to >=1 category at once, you have a set of binary variables (each variable = category; 1=belong, 0=no). Compute one of the [numerous indices for binary data](http://www-01.ibm.com/support/knowledgecenter/api/content/SSLVMB_20.0.0/com.ibm.spss.statistics.help/syn_proximities_measures_binary_data.htm) and submit the matrix of the index values to hierarchical cluster analysis.
A few other clustering criterions can offer this opportunity. See, for example, Gap statistic.
(+1) For sharing your solution.
Of course, so simple. Thanks
Replace lambda value with 1/6 instead of 6 which is the mean. Wrong formula corresponds to wrong result!
At the end of that blog post it says "instead of using the uniform distribution as a prior, we can be even more agnostic. In this case, we can use the Beta(0,0) distribution as a prior. Such a distribution corresponds to the case where any mean of the distribution is equally likely. In this case, the two approaches, Bayesian and frequentist give the same results." which kind of sums it up really!
It would be the class but you should not try to draw inference to the population from the sample.
Glad they were of some help. Furthermore: No, be careful; `model complexity` is definitely not similar. If anything it is an open question (in technical, theoretical as well purely philosophical terms) how many degrees of freedom one random effect should account for. Eg. Should you threat in a $N$ = 10000 sample, a random effects with 3000 levels as conveying the same information as one with 25 levels? Intrinsically I would say "no", though ANOVA would treat them as exactly the same. (I am not saying that my "no" is the correct answer, I am just commenting.)
Ah. I think it's the proof 2 in that web site the professor wants. Thanks for the link!
This is a variant of http://stats.stackexchange.com/questions/12174/time-taken-to-hit-a-pattern-of-heads-and-tails-in-a-series-of-coin-tosses and is solved using the same methods.  (I'm sure I had seen this question asked and answered on Math.SE around the same time.)  Regardless, SE requests that you post only a single copy of your question at a time.  If you would like your question to appear here, then please ask the Math mods to migrate their version to this site.  In the meantime--although this is not strictly correct--I have closed this version as a "duplicate."
You are interpreting $X$ and $X_n$ as deterministic quantities, not as random variables. So the sentence "If $X_n=X$ then $|X-X_n|=0$ for all $n$'s" does not make sense. Those are random variables, e.g. $|X-X_n|=0$ with probability $1-(1/n)$ and $e^n$ with probability $1/n$. From there, you can conclude about convergence in probability and $L_2$ [lack of] convergence.
Thanks for the response, I was worried that this might be the case. +1 for elucidating the bounds so clearly - if no one else has answered by the end of the bounty period, then I'll give the tick to this answer. Cheers.
That is cool - I had no idea that existed.
+1 for R solution!
Here are a couple of other questions of possible interest: [is-normality-testing-essentially-useless](http://stats.stackexchange.com/questions/2492/), for a discussion of the value of normality testing, & [what-if-residuals-are-normally-distributed-but-y-is-not](http://stats.stackexchange.com/questions/12262/), for a discussion / clarification of the sense in which normality is an assumption of a linear model.
Are the people in the 2 groups the same group or two independent groups? What is(are) the group size(s)? You mentioned that the 12 items will yield a score, what is the distribution of the score?
Have you tried the package "Oblique.tree" ?
http://cran.r-project.org/web/packages/oblique.tree/index.html
best
In all fairness, the conditional independence assumption is violated in nearly all cases where naive bayes is used, and the technique often performs well anyway. That's why we call it "naive," because we know the independence assumption is generally not justified.
BCa intervals are "better" than other types of bootstrap intervals in terms of coverage. This is, their coverage is closer to the nominal value. In some scenarios this implies that the intervals are wider or narrower than other types of bootstrap intervals. Regarding your last comment, the length of confidence intervals converges to $0$ as the sample size $N\\rightarrow\\infty$. Intuitively this happens because the larger the sample is,  the more information it carries about the parameters and this is reflected on the "precision" of the estimation.
Maybe this will help: http://stats.stackexchange.com/questions/23117/which-mean-to-use-and-when
Thank you very much for your answer, I'll check how it behaves with a [qqplot](http://www.statmethods.net/stats/rdiagnostics.html) or is there a better method that you suggest? Part of my question is in regard to *very large* series, when looking at the central limit theorem almost the underlying distribution quickly disappears when n is large. In registry studies the n is > 30 000 and therefore even if the qqplot would suggest some non-normal distribution I wonder if this really matters?
@EpiGrad - It took me a few error messages to actually realize what they were telling me. Next time, I will give myself a bit more time to understand the error before coming to SE.
Thank you.From some online material for a masters of applied statistics course.
Nothing strange to me. Large odds ratios and poor specificity, this happens a lot in my field (genetics).
I agree, a very nice book. But I would love to see another refresh (last ed was in 1999). Most recent treatments don't stack up IMO.
Thank you, I clearly have some reading to do.
Could you please add references to the measures you mentioned?
Thank you cupoftea. Indeed we had previously read the reference you mention. It thoroughly discuss our issue, especially when the number of experimental units is low, but we would like to know whether there is previous experience specifically in forest ecology studies. Also we are seeking comments on the initial conditions question ([reference](http://books.google.it/books?id=iu_PZYpjFJ0C&lpg=PP1&hl=it&pg=PA14#v=onepage&q&f=false) p.14) and whether our approach was correct.
How about `glm(visits ~ x + chicks, offset=log(time), family=poisson)`?  The offset is there because you want to estimate the effects of x on the rate of visits rather than the count of visits.  Chicks don't change that.
As for the books in the tag, I have Venables and Ripley (2002) and Gelman & Hill (2006) and haven't found anything there. I also just got Pinheiro & Bates - Mixed Effects Models in S and S-Plus from the library - there are a some pharmacokinetics datasets there, but nothing very close to what I'm looking for.
By 'difficulty in responding' I meant the effort it took to respond. When it takes more effort to respond, you are more likely to get extreme values.
There's a lot of literature on mode of survey administration, but it's been a while since I looked at it and when I did it wasn't about this kind of survey
This question appears to be off-topic because it is about troubleshooting SPSS. Unfortunately with the purchase by IBM SPSS has taken away some of the older plug ins. This [recent post on the SPSS nabble list-serve](http://spssx-discussion.1045642.n5.nabble.com/matching-versions-of-R-and-SPSS-td5717805.html) states it is not available anymore for versions pre 18.
Well the term in $y$ is just constant and comes out the front. Then I'd be inclined to do a transformation to say $w = \\sqrt z$, so that the term in the integral simplifies somewhat, leaving you with a quadratic form in the exponent, and you could then try to play 'spot the multiple of a density'. But generally speaking, an integration problem is probably best placed on math.SE. If you'd like to flag it a moderator should be able to move it for you.
Sorry if I've misinterpreted but are you suggesting using the likelihood ratio test? Except in special cases, the GEE estimator is not an MLE, so you can't use the likelihood ratio test there.
McNemar's test is for nominal data.  For ordinal data, as here, people often choose the [Wilcoxon signed rank test](http://en.wikipedia.org/wiki/Wilcoxon_signed-rank_test) or [sign test](http://en.wikipedia.org/wiki/Sign_test) (but the power of the latter tends to be low).
Any ideas out there?
I was thinking along the same lines, but this is not the product of two random variables; it's the sum of a random number of random variables. Those conditions have different variances (though they have the same expected values).
The se.fit that predict.glm produces is a standard error for the *mean* prediction. For some GLMs it's meaningful to talk about a prediction interval (e.g. for the normal and Gamma), and a standard error for a future observation, but even in the cases where it makes sense, the problem - while easy for the normal - is difficult in the general case. You can do (for example) an asymptotic simulation -- simulate from $\\hat\\eta-\\eta$ and then from $(Y|\\eta-\\hat\\eta)$.
@John Thank you for your answer. It is very helpful. I will follow the procedures you outlined. If the AIC test favors the two-level categorical approach it will improve the overall quality of the analysis greatly. My apologies for the vagueness of the question.
Hi Rozlin. Welcome to the site. I've edited your question to use LATEX formatting. Please double-check that it still reflects your intended meaning & that I haven't accidentally introduced any errors.
Ah, a philosophical rather than a technical point!  I'm OK with that.  You might want to move your last sentence, or a variant of it, into your answer, just to expand it some.
No, I think you are confusing the definition of an MA(n) model, where the regression is only in terms of the $e_{t-i}$'s, with its estimation, where the $e_{t-i}$'s are estimated from the data.
However, you should modify it to ask about degree of dependence rather than of independence (dependence has degrees, independence is sort of 'it is or it isn't').
That link contains no data or examples and the links from it do not work. Please endeavor to make your question stand by itself by including (a representative subset of) the data in the question and/or linking to an image.  (If the link is legitimate, community members can embed it into the question itself on your behalf.)  When you do this, could you please also add a few words explaining *why* you want to determine the number of modes. What conclusions will you draw from that?
@tomka: If you have a biased estimator at hand, bootstrap sometimes helps to get a better estimate of the parameter of interest. This bootstrap bias corrected version equals twice the statistic minus the mean of the bootstrap distribution.
I don't understand why this question is upvoted so much while good statistical ones go under the radar. This question is about a software error because of a user mis-usage...
I understand what you proposed. I just don't get why `zeroinfl` doesn't support formula like `y ~ 0` ...
Please don't copy and paste homework assignments verbatim. Put some effort into it and show what you have done so far. As is the last sentence doesn't even make sense.
Interesting, now I get predictions but the number of predictions is all wrong. It seems I get for the training set not the test set that I am giving to the function. I only gave it testX and not testY though. But what I wanted to do was to get predictions for a test set not for the dataset that I used in training...
The main difference between these two models is in stationarity of series. Unless I have a reason to believe there's nonstationarity, I'd go with ARMA. For instance, if these were monthly changes in savings account in dollars, I'd expect these to be nonstationary.
this is the moment generating function $m_{X}(t)=E[\\exp(tX)]$ evaluated at $t=-1$.
Welcome to the site. 

One note: Usually observations are rows and variables are columns - just something to be aware of when you write a program to do whatever you wind up doing.

One problem I can see is that, with so many more variables than observations, there will doubtless be **many** sets of variables that perfectly identify the subjects. Even your example linear model has 4 variables, which is probably overfit. Can you get more timepoints? If not, maybe look at each variable individually.
@GavinSimpson My area of specialty is Gaussian Processes so spatial moddels in ecology are probably my biggest area of interest although to be honest I am not 100% savvy on all the topics out there so an intro book would be just as interesting to me.  Code or theory books are also welcome, I guess I am more so looking for interesting topics of research.
@Preston Thayne: Since the standard deviation is *not* the expected value of `sqrt((x-mu)^2)`, your formula is misleading. In addition, just because squaring has the effect of amplifying larger deviations does not mean that this is the reason for preferring the variance over the **MAD**. If anything, that is a neutral property since oftentimes we *want* something more *robust* like the **MAD**. Lastly, the fact that the **variance** is more mathematically tractable than the **MAD** is a much deeper issue mathematically then you've conveyed in this post.
I think this is a too broad question. Because almost all statistics can be used in data mining, I don't see any reason for this question to exist.
Since $\\lambda t=E[X]$, I could do regression on that instead, but I'm not sure if that's valid. It's seems like the error in OLS to find $\\lambda t = X$ would be different than in an actual Poisson regression.
@ttnphns--Thanks for your response!  I like your corrected diagram.  I have simply excluded these two methods from my analysis...but out of curiosity, do you have a reference for your last point re: geometric methods and the nonmetric distances?
Several thousand is small computationally maybe you need to profile and analyze your code or consider parallel computing
Is this from a randomized trial? Why is there baseline imbalance?
And thanks again, ttnphns. I think I will try using a Mann Whitney U test.
linear model is nested in mixed, difference between them is only random effect
From the example it looks like McNemar is what I need, but I'll keep Fisher in my mind too. There are just too many similar tests for a beginner to comprehend :)
Many of us probably don't know what perplexity means and what aperplexity plot shows.  I know I don't.  Could you enlighten me (us)?
thanks. I think it was in reference to a normal distribution where they compared mean to the SD.
@Macro The formula I gave for the SE depends on bivariate normality. I guess I should say 16 subjects instead of items? Participants?
You forgot the ANOVA of `B.lme`. Or rather, you have an ANOVA of `A.lme` but it looks like it's for a `B.lme`.
Connect them and look at the time series plot and then see if you need that Box-Cox transformation or not.
I guess I should remove it... poor Karl Pearson, one of the inventor of hypothesis testing not understood by the 21st century ... I would vote up if I could, but it's me that put it here :)
What domain is this? This seems like overly ornate notation for a least squares problem.
Thank you for your advice.  I want to compare forward selection against several other methods, including elastic net, on simulated data sets.  So I still need to address my original question of how to handle p > n in the Sequential Sum of Squares test.  Anyone?
So "normal" means data consistent with some underlying probability, $p$, of being 1200, and "abnormal" means data that came from / were generated according to a different underlying probability, $p'$, is that right? Moreover, you are really only interested in if $p'<p$ (ie 'less than "normal"')?
One way is to plot the mean of fitted values from your model, conditional on values of these predictors of interest. You could make the x-axis one of the predictors, and then use a different color for different values of the other predictor (with the y-axis being the mean of the fitted values).  You can do this with R basic graphics, lattice, ggplot, or other packages; e.g. http://www.jstatsoft.org/v08/i15/paper
Finding the best set of 5-10 columns would require some kind of algorithm (to select those columns among the 200). Many options are possible. But to evaluate the quality of the fit for any 5-10 columns, the R^2 from the linear regression would be an excellent metric to look at (being the squared correlation between the dependent variable and the predicted value); and the coefficients from this regression would maximize this fit.
This question needs more context/information to answer. Are you interested in "what's the probability that X is just a typo for Y?" or in doing entity resolution (e.g., "the 44th US president" == "Barack Obama")? What prompted this line of inquiry?
@gung: Yes, you are right: I was confused by the same terminological mistake in the original question. I will correct it now. Regarding your additional question: actually I don't know what the best practice is. I would be very interested myself if you could briefly explain.
"AB=0" is not a test statistic. You need a purely sample-based quantity. Consider something basic like the mean as an estimator for $\\mu$. Could the mean itself be a test statistic? What's its distribution under $H_0$? What would tend to happen under the alternative? Can you modify that in some way to make it more useful?
SPSS Statistics (formerly SPSS) is a general purpose statistical package. SPSS Modeler (formerly Clementine) is a data mining package. Knowing that you can deduce differences.
@FrankHarrell I've edited the question to clarify that I'm trying to assess whether the data meet the model assumptions. Thanks for the introduction to partial residual plots, I think these are exactly what I'm looking for.
Can't find any good sources on nonlinear sandwich formula (only simple linear regression). Is there some kind of readymade "follow these steps" method?
It's not *data* that's nonparametric. Why are you testing normality?
OK, I used the cross-validation and received 80% accuracy. But how can I be sure that in the implementation of no small mistake? And if it is correct, you get 90%.
Important point, and it can be extended: Many models for ordinal data hinge on the idea that it's not the ordinal data but their cumulative probabilities that can be modelled.
interesting point +1
What about I use neural network on this data ? any neural network package I can use in forecast ? Thank you
Thank you Nick, that was very helpful. I suspected as much (and summarized the factors), but was confused: In an earlier analysis using SPSS (a few years ago), I'm relatively sure my extracted factor had the "right" scale (meaning something like 1-7). But that was a EFA, so whatever. I'll do a few days of closer reading before I start with actually writing anything down. Thanks again!
No serious harm; just against site policy, hence the collective cleanup action. Hopefully we can just convert this to a comment for you, or if not, hopefully you'll find other ways to help :)
Are you suggesting to export charts from SPSS as .eps files? How would you do that, what does it mean to choose "graphs only" / what menu is that in?
The reported AIC values may also be non-comparable because different constants are omitted.
I read the first part of your answer and think it's promising. Two quick notes -- this was multiple choice so probabilities of guessing correctly are 25% or 20%. You are correct in that we may assume a subset of questions was leaked before the exam. Will return to this on Sunday or Monday.
Is the hand assigned to treatment chosen at random? Is the outcome variable related to handedness (i.e., whether you prefer left or right hand)?
Double check `mod.m=8/3`. Because if you set `mod.m=2.5714`, then they seem to be identical.
@RomanLuštrik If there is an error in my calculations, I would think it to be in the code: `ds.group$sd.prop <- aggregate(Ratio.sd ~ Group, data = ds.Agg, function(x) { sqrt( sum((x)^2) ) / length(x);})$Ratio` But I am certainly not sure.
Note the definition of the ecdf (or the cdf more generally); `ecdf(b)-ecdf(a)` would estimate $P(\\text{a}<X\\leq \\text{b})$. Of course with continuous variables the distinction between $<$ and $\\leq$ is unimportant. If $X$ is discrete, then you can *estimate* $P(X=7.5)$ by computing the sample proportion of values that are 0.75.
(+1) that's a good note regarding SPSS forecasting the constants, no surprise that then you have MAPE $0$.
Any progress here?
Are the 8 different people than the 30, or are the 8 part of the 30? Is it the smallness of the 30 or the smallness of the 8 that's of concern?
I use the matlab geornd function. so what i am doing is X=geornd(p) + s