tttjjj/elt_bench huggingface dataset:
I almost didn't bother to comment since it goes without saying that @mbq has been a great moderator. In particular, in pushing forward the journal club.
Once the students have seen the questionnaire once, it makes no sense to randomise the order of the questions, because they already know what the questions are. So you can only fix the order bias in the first question. My comment above suggests how you can prevent teacher order bias.
I'm not an expert on time-series, so someone else will have to help you, but I suspect what you want to do is [vector auto-regression](http://en.wikipedia.org/wiki/Vector_autoregression), there are also a couple of questions on that topic already on CV which you can find by searching on the [tag:var] tag.
@proton, it is certainly possible for 2 distributions to have similar qq-plots, but you will have *exactly* the same problem w/ a statistical test. When distributions are similar & you have few data, you will have low power.
Currently all the analysis is based on just RMSE calculation. I have the whole Netflix dataset, randomly take-out some particular percentage of data, then impute it and estimate RMSE. This procedure is done 1000 times in order and the results are averaged to avoid bias. I thought that since the dataset is huge, I am kind of guaranteed from overfitting and all corresponding staff? is there something else which should be considered during imputing? or all the measures that you mentioned must be used?
We'll need you to describe what you mean by "the statistical significance of a metric."  You haven't yet posed a statistical problem to be solved.
@Spring23874 Can you show us any attempts you have made at this problem or where you are stuck in trying to solve it?
@gung I believe most people would include the calculation and reporting of data summaries within the scope of statistics.
Thank you. I was referring to real world events as "Targeted" and "Observed" for data retrieved form the simulation. I will also try asking in stackoverflow then. There had been no good way for comparing real world with virtual world in Virtual Reality for years. Most research was evaluated by just using user based liket-scales or similar methods. I think this should be another researach. I will look into your link and update this for everyone. :)
"A literary work" is only an excuse for misrepresenting reality if what Taleb wrote was a novel. Not going into a technical treatment is excusable, misrepresenting something wholesale is less so.
It is possible to do MDS on 6000x6000 matrix, but it will take time (especially nonmetric MDS).
About the strongest you could say would be something like - "The Q-Q plot is reasonably consistent with normality, but the left tail is a little 'short'; there's mild indication of skewness."
"xspline"  is more cute.
Thank you, I added scatterplots of PCs and variables. Not sure if I have such clusters, I will try to check.
@are you using `gam` or the like (`mgcv` package)? If not, you should check it out.
@Peter Ellis: Yes, good point!
@Glen_b Biased in what way, though?  I don't think this necessarily follows... Consider 2 predictors, x1 and x2, s.t. x1 = x2 + (very, very small noise), and a truth function y = x1 + x2.  OLS might return you something like y = 3000x1 - 2998x2, and RR will probably give you something close to y = x1 + x2.  In my poor understanding of bias, I don't think either of these would give biased estimates...
Could you tell us what *you* mean by "predictive power" and what "better" means?
This is pedantic but the residuals are not the errors. The residuals are the observed differences from the estimated model, $y_{i} - x_{i} \\hat{\\beta}$. The errors are the differences from the true model $y_{i} - x_{i} \\beta$.
@cryptron, the key question is what is a **sound** threshold.  For example if each raised alarm and non-raised alarm incurs certain costs, the threshold can be chosen such that minimises total costs. For that we need cost data. Without the exact definition of **sound** it is impossible to measure how to evaluate the method chosen for picking the threshold.
I'm a little confused about what you are looking for in an answer. A citation to a paper that uses a conditional multinomial logit in a recommender setting?
can you supply the data?
Concerning your first comment, I direct you to this [discussion](http://stats.stackexchange.com/questions/86835/normality-assumption-in-linear-regression). The question in your second comment is now a pure programming question that should perhaps be moved to stackoverflow. Please update your question to specify what is being asked.
It does seem to me that the first issue is one for research, and might not have "best practices" yet. It may very well be that a solid introduction to basic testing and drilling in the multiple hypothesis problem might be the best place to start.
I have never heard that too and it's hard to guess without the context. However, when you choose a prior on $\\theta$, then you implicitly assign a prior on $\\theta^2$ (for example). Perhaps this is the what the author means ?
This is an excellent suggestion. I tried various LaTeX hacks for Evernote, gave up and have settled on `org-mode` and exporting to LaTeX. For paper management, I use Zotero though, which is able to export bibliographies as BibTeX files.
Frobenius-Perron theorem is what you probably seek for?
How is text "continuous data"? Please explain.
Depending on the model, your posterior might be very close to improper using this prior
The point that I didn't appreciate previously is that the Box-Cox likelihood profile is not invariant to scale changes ...
Curious what others may answer. But you seem to be comparing two completely different commands and surprised they don't give the same answer.
You might probably want to add that `cv.glmnet` returns two values for $\\lambda$, `lambda.min` and `lambda.1se` (“one-standard-error” rule).
Just one pint out of 20 is defective.
According to the author of Encog one output node always returns result 1, thus for Encog two classes are need when SoftMax is used.
They're called parallel coordinate plots: http://en.wikipedia.org/wiki/Parallel_coordinates
@Néstor: I'm watching the live broadcast of the Higgs press conference now, and no one is mentioning Bayesian interpretations. "p-values" and "significance level" are used, but only horribly misinformed Bayesian would interpret those as probabilities that the signal is random noise. I think that the text in the quote in the OP's question simply is a misinterpretation of what a p-value really are.
@Dimitrij Price is not an independent variable: it's the *dependent* variable: "I would like to understand how to model prices for any car based on this base information."  I fear that with this misapprehension you may be taking @murrekatt very far afield.
Thanks, this is exactly the kind of information I was looking for.
An author has little choice about  the typeface a journal is published in. Many probably won't even know.
thanks,
Categories have values from 1 -9. And my aim was to guess category from independent variables.  
So I am confused about what to do. Should I do classification or regression? Can you please tell bit about how to perform classification in r-tool?
However also please have a look at my question here:
http://stats.stackexchange.com/questions/43040/verification-of-a-regression-model
Great explanation. Thank you. BTW I am not referring to linearly increasing hazard rates rates.
Actually you have made me think of a good idea. I can impute the last value and include that line with the current plot as dotted. This would seem to achieve my goals. Perhaps someone else has a better idea.
you should use ANOVA (analysis of variance), or its non-parametric equivalent according to variable distribution. Can you tell more about your data, or post it here? (maybe you have repeated measures which must be taken into account)
@Kirk You can create it yourself in no time.  It's set up so by typing the formulas in C2:F2, you can drag them down as many rows as you like.  Then type the LINEST formula in I2:L2.  Just five simple formulas and you're done.
Hi, this is [Fisher-Neyman factorization theorem](http://en.wikipedia.org/wiki/Sufficient_statistic#Fisher.E2.80.93Neyman_factorization_theorem)
If you have a numerically singular model matrix, most regression functions will omit variable or levels until they are able to return <something>.  Try putting the `cq` term in explicitly.
@ Stromgren If I understand correctly from your comments below, this is a joint [`P(Kim,US)`] versus conditional [`P(Kim|US)`] probability question. I would suggest that you incorporate some of your explanations in your comments into the question to increase clarity.
@Eoin correlation is unsuitable as a measure of 'how well things match'. For example if the second machine always counted only 10% of the first machines values, they'd be essentially perfectly correlated, but very different in value.
In principle I agree with the objections, this is why I asked the question in the first place. But the problem is more about how to translate the prior information on prior distribution. For example if I know that the parameter is positive, maybe a scale parameter and lower values are more likely, what should be my distribution?
Take a look: http://xianblog.wordpress.com/tag/constrained-parameter-spaces/
Did you want to address the issue of a significant interaction with no main effect?  I thought that was one of the key issue to the OP regarding his analysis.
(+1) Glen. To @BYS2: There are a couple of key points here. First, if we have a pivotal quantity at hand, it provides a very convenient means for constructing confidence sets, but they don't often exist. The whole point of a pivotal quantity is that the distribution depends *purely* on *known* quantities. Second, the pivotal quantity is intimately tied to the underlying model. If the data deviate from the assumed model, then the distribution of the test statistic may as well and its characterization as a pivotal quantity may not be quite as relevant. :)
See, for example, the discussion [here](http://stats.stackexchange.com/questions/89624/linearity-assumption-in-ols-with-dummy-variables/108976#108976)
If the interaction term is not statistically significant, then it means that you don't have enough evidence to conclude that the effect of EF on your outcome is different between your two Diagnoza groups
Do you mean that x, y and z are co-linear?
In R 2.15.2, `binom.test(no.acc.w,no.w, p = .304, alternative="two.sided", conf.level=.95)` works. The error message is because you have two arguments, both named `p`.
Related: [Using t-test for comparing Likert responses before and after intervention](http://stats.stackexchange.com/q/20245/930), [How to compare Likert scale items before and after an intervention?](http://stats.stackexchange.com/q/59625/930)
It is not true that stochastic gradient descent doesn't really converge and just wonders around a certain point. That would be the case if the learning rate was kept constant. However, the learning rates tend to zero because in this way, when the algorithm is close to the minimum of a convex function, it stops oscillating and converges. The key of the proof of convergence of stochastic gradient are the conditions imposed on the the series of learning rates. See equations (6) and (27) of the original paper of Robbins and Monro.
If you want to assume normally distributed data with a fixed variance from the model, then it is easy. The difference is also Normal with twice the variance but with a mean given by the expected shift (from the slope). The statistic you seek is then just the Error Function suitably shifted. Relaxing any of these assumptions will change the result but it is hard to describe this in general.
A nice way to visualize three values summing to a constant is [ternary scatter-plot](http://en.wikipedia.org/wiki/Ternary_plot). But you have _four_ not three proportions adding up to 1. This makes ternary plot not applicable... but who knows? Maybe there is a way out. Ternary plot can be 3D which allows to add one additional feature. Of course it can also be coloured/poit-sized/matricized to show more continuous or categorical features.
Can you upload an example of AMOS' visualization tools of which you're so fond?
+1, nice answer! Welcome to the site, @tabSF. You'll be happy to know that CV supports $\\LaTeX$ via mathjax. I took the liberty of tweaking your answer for greater readability; make sure it still says what you want. Since you're new here, you may want to read our [FAQ](http://stats.stackexchange.com/faq) as well, which covers the markup options available (among other things).
Nice answer. Also, the price of a stock doesn't mean much on its own. Typically you would have to scale it by something like earnings or assets to give it a meaning. This is why financial economists invariably run regressions with returns.
I think this question is too vague and broad for anyone to supply useful advice. What is the structure of your data? What observational units are your data measured on? Which variables are you interested in investigated relationships between? Significantly edited and added to, I think this question might be good enough to justify migrating to stats.SE rather than just closing.
Does the book come with a video?!
It looks like the leaves end up at zero. I finally (and accidentally) found confirmation of this in the docs: http://publib.boulder.ibm.com/infocenter/spssstat/v20r0m0/index.jsp?topic=%2Fcom.ibm.spss.statistics.help%2Fidh_clus_plo.htm and click on the dash-underlined "dendrogram. If you want to edit that into your answer, we'd be all set here!
@user48700 Could you summarize how *Statistics Without Tears* explains this?
Is the question of interest "is the new one better, no matter by how tiny an amount, as long as it's more than could be explained as random variation?" or is it more "how much better is it? Is it enough to really say we made a difference?". The second is really about measuring an effect (and would lead to say an interval estimation exercise rather than a hypothesis test).
Edit: looks like one solution is to split the data into:
presence/absence binomial (Bernoulli) BRT
+
log-transformed presence only gaussian BRT.
As per: www.sefsc.noaa.gov/sedar/download/S34_WP_12_Froeschke and Drymon 2013.pdf?id=DOCUMENT
Thank you for your help. This looks like good information for my work. Just to add some information to this discussion for others who may have similar study designs. Mielke & Berry have a book "Permutation methods: a distance function approach" (2007) and some software for different analyses in the book at http://www.stat.colostate.edu/~mielke/permute.html
When these variables are the *dependent* ones, this is a well-known problem of censored regression.  The accepted practices are far better than the first two options, which have been discussed extensively in many fields and found to be wanting (they are biased, sometimes strongly so).  It doesn't usually require "elaborate models."  For example, this problem can often be recast as one of survival analysis and it can also be tackled directly using maximum likelihood.
@Srikant, thanks for the suggestion.  I spent some time trying it out and it seems like a good start at getting a quick visual comparison, especially when the fit is bad.
You want to figure out the probability of a certain class, given that it contains certain words. Whether you call that class "spam" or "announcement I would like to see" isn't relevant at a mathematical level.
You can achieve perfect (100%) power with a procedure that always rejects the null hypothesis no matter what the data are.  The right way to formulate this question, then, is *given a specified test size,* can the power be increased with a more sensitive measure?
the only thing i could find is using proximities but that's not related to unsupervised.
I would just add that if you want to be fairly conservative in your coverage, the best CI method to use might be the Blaker interval, which is not mentioned in the Wikipedia article @Roland linked to, but is implemented in R with the package BlakerCI.
@fRed: nop, here is the paper, Coifman and Donoho: http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.125.3682&rep=rep1&type=pdf
@Rodule: How's it goin'?
The $p$-value of a test is actually the probability of obtaining a statistic at least as extreme as the one you've observed. So, assuming $A$ and $B$ are equally matched, what is the chance that $A$ beats $B$ $31$ or more times in $50$ matches? That is the value you're comparing to $\\alpha$.
Unlike the example in your code, which has *one* independent variable (bid) and therefore plots as a curve in a (bid, probability) plane, the example in your edit at the end would not plot as a curve: it would be a *three* -dimensional hypersurface plotted in four dimensions (bid, bid2, bid3, probability).
(I was thinking of multivariate $x$.)  Writing $A=(a_{ij})$ and $x=(x_{kl})$, compute $$Axx^\\prime A^\\prime =a_{ij}x_{jk}x_{lk}a_{ml}=(a_{ij}a_{ml})(x_{jk}x_{lk})$$ (sum over all repeated subscripts). Writing $\\mathbb{B}=(b_{im, jl})=(a_{ij}a_{ml})$ and $(y_{jl})=(x_{jk}x_{lk})$ and considering $im$ and $jl$ each as single indexes, this is the matrix product $\\mathbb{B}y$ and the expectations of $\\mathbb{B}$ and $y$ can be taken separately.
@ayush Ah, and I just realize that you **never** vote up any of the answers that were provided to you (though you have enough rep now).
It is doubtful that your sample size is sufficient for assessing interactions.  And if you were not able to pre-specify two or fewer interactions this is particular problematic.  Searching for interactions in my experience results in findings that are unlikely to replicate.  Problems include low precision, low power, interaction terms co-linear with omitted main effects, interaction terms co-linear with each other.  Since your interaction search is not hyothesis-driven, if you do want to assess several interactions you need penalization (shrinkage).
The one-sample t-test with $\\text{H}_0: \\mu_0=0$ is performed like so: `t.test(x)` (assuming your data were in `x`). It returns a test statistic and p-value in the output.
@cespinoza : Ronald Fisher : To call in the statistician after the experiment is done may be no more than asking him to perform a post-mortem examination: he may be able to say what the experiment died of.
Is drawing two times 5 valid?
It depends on what you mean with "best classifier", if your task is building a classifier with good accuracy overall, I would choose FREQENT. On the other hand, if, like in the most of rare class classification tasks, you want to classify better the rare class (that could be the "negative" or the "positive" class) I would choose MAXINFOGAIN. I think your learning curves interpretation is correct: with 100 features you have bias and you may add them, with 1000 you have variance and you may remove them. Maybe you can try a trade-off between 100 and 1000 features to get better results.
Hmm, interesting: downvote on my answer without courtesy of indicating what is wrong. And the other answer here is effectively the same as mine, but hasn't been downvoted.
Are you trying to predict lifetime value of a customer? Or are you trying to predict whether a customer will renew their licence? They sounds like two very different things to me.
Thanks! I found the first one helpful!
An alternative method is to click the chain link icon just above the field where you type your answer, paste your url in the box that appears, then the code Procrastinator wrote will automatically appear in your answer, and you can type in the link text right there.
This is a helpful answer (+1) but be a bit careful with you coding example: 1. You need `plyr` to use the function `ddply()`, 2. you call `ddply` on a dataframe that doesn't exist ($d$). On a different note: while you correct say that you get the same $t$-values, you need to draw attention to the fact that defining the related degrees of freedom for that $t$-distribution is not a well defined aspect and people should not be tempted to naively extract $p$-values from it. Same $t$-values do not lead to same $p$-values in this case. (Nice answer nevertheless)
A working example appears at http://gis.stackexchange.com/questions/35159, where weights are based on distance in the plane.
@daemonfire300 Nope. Both the median and arithmetic mean are measures of the central tendency of the distribution of the variable of interest. Usually, observed values appear on the x-axis in an histogram, with the y-axis representing frequency, counts, or density of those values grouped into bins of equal width. (In case we use unequal bin width, the area of each bin will represent the frequency, not the height of the bar.)
Yes it does. Thanks!
Because RMSE and MAE are two different measures of error, a numerical comparison between them (which is involved in asserting that MAE is "lower" than RMSE) does not seem meaningful. That line must have been fit according to some criterion: *that* criterion, whatever it is, must be the relevant measure of error.
@RasmusBååth Well, you are not using the method of moments precisely. The expression `b^2/((a-1)^2*(a-2))` is actually the theoretical variance. The method of moments consists of solving the system $\\dfrac{\\beta}{\\alpha-1}=\\bar{x}$ and $\\dfrac{\\beta^2}{(\\alpha-1)^2(\\alpha-2)}=\\dfrac{1}{n}\\sum_{j=1}^n(x_j-\\bar{x})^2$ in terms of $(\\alpha,\\beta)$. After this, you could plug the solutions $(\\hat{\\alpha},\\hat{\\beta})$ into the expression for the variance $\\dfrac{\\beta^2}{(\\alpha-1)^2(\\alpha-2)}$ in order to obtain an estimator of this quantity.
Incidentally, it's common to interpolate in the *reciprocal* of the degrees of freedom. This tends to improve the accuracy.
NATO's technical report really helped me along. It's a great reading on the subject. Thanks!
Nice explanation, thank you very much!
Have you seen the [UCLA test decision tree](http://www.ats.ucla.edu/stat/mult_pkg/whatstat/)?
This appears to be a standard textbook-style question. Could you please add the `self-study` tag, and read its [tag wiki info](http://stats.stackexchange.com/tags/self-study/info)? With that in mind, what have you tried so far?
This is not a forum; please use the edit feature to alter your posts.
I like this solution! One concern I have is that given so many possible permutations, wouldn't there be a lot of local minima to get stuck at?
How does this apply to spatial data?
I think it would be more on topic for academia.stackexchange.com.
It's impossible to answer this question in general. The error measure you use should represent the cost of that error that you are trying to minimize. If you only care about hits/misses you could use something akin to classification error: $\\sum_i^n I(y_i \\neq \\hat y_i)/n$ where all errors, large or small, count the same. What is the underlying problem you are solving?
Could you share with us the reason you need a single index (especially since you plan to use it in multiple regression, which by definition allows inclusion of more than one explanatory variable)?
@whuber correlation= cov/(std.dev1 * std.dev2). But I do not get your point?
One of your biggest concerns could be the possible nonlinearity of the relationship due to distinctly different responses when the IVs are exact zeros compared to otherwise. One way to assess (and cope with) this is described in my answer at http://stats.stackexchange.com/a/4833. It proposes introducing additional variables that are indicators of zeros. (You do not necessarily have to take logs of the original IVs as well, which was discussed there only to respond to that particular question.)
I am guessing it is a typo or an oversight.
You might want to add why you want to transform these explanatory variables. They do not need to be normally distributed for OLS.