tttjjj/elt_bench huggingface dataset:
Nice answer.  You might also note that if the MLE is found through a numerical optimization, then often a numerical estimate of the Hessian (matrix of second derivatives) is available (ie, for newton or quasi-newton algorithms).  In this case, a cheap and dirty estimate of the information, and hence standard errors, is this numerical Hessian.
(Incidentally, I believe my answer is addressed to your question as edited. Let me know if there's sill some lack of clarity.)
@John could you delete your comment and post your question as a separate question on this site (i.e., using "ask question up the top". If you feel that your question is related to this question, then add a link to this question in your new question.
what do you call 'this'? I used just a few computation tricks, I did not know the result before. For a general reference I use the book related to [this site](http://www.stat.pitt.edu/stoffer/nltsa/), because my teacher is one of the authors.
No! The adjusted p-value increases by a factor of three over the raw p-value.  The way you are probably looking at it is in reverse namely what should the raw p-value be so that i can get an adjusted p-value less than alpha.  Then you require the raw p-value to be alpha/3 or lower.  I am saying that if you are given that toraw p-value is r then the adjsuted p-value is 3r and it is 3r that is compared to alpha.
I actually found some other posts by the user Irishstat, which pointed to some papers that go about it a bit differently. I'm going to dive into those first and then look at your suggestions, thanks!
@DrWho A) You didn't answer my questions. B) Yes, it is a complex issue. It is therefore probably not wise to expect a complete and stepwise solution posted on the web. This is a question for which you really need to hire a consultant (someone who knows time series) and work with them over a fairly long period to figure out what you want to do.  You can get some advice here (I have certainly gotten good advice here!) but in this case, you need more than what will fit.
Agreed...Is there any other way to estimate the parameter other than MLE technique in logistic regression ?
It depends what you mean by complete. If it means that you have the entire population, then your parameter estimate is no longer an estimate. This assumes, of course that the population stays constant. And a word on the careless language here: a parameter has an *estimator* and a particular value of the *estimator* is an *estimate*.
I think it really depends on your goals.  Are you trying to fit a certain model, or generate plots, or make predictions about the future?
Great recommendation, thank you -- I got a copy recently based on this and it really is quite good.
Why do you suppose an objective function has to be differentiable?  And why do you (implicitly) assume all M-estimators are based on differentiable functions?  I think a closer examination of these assumptions may lead you to the answer you seek.
The Mann–Whitney test is for two groups, not three; I think you'd want the Kruskal–Wallis test instead, *if* you know what your dependent variable is. You haven't made that much clear. How do you intend to use your 48 Likert ratings? Are you sure your way is valid, or are you unconcerned about it? If you were planning on using a simple sum or average of your Likert ratings as your score, you probably should be concerned about the validity of that score. I also recommend clarifying your point about "stratifying questions" – I don't understand what you're proposing, but maybe I'm being dense...
A "unit," by definition, is $1$.  Thus it would be one-tenth the change from $x=0$ to $x=10$ or one-fifth the change from $x=10$ to $x=15$.
See [this](http://stats.stackexchange.com/a/20856/32036) on the matter of stepwise selection...It's not exactly a safe recommendation.
A good book on this, at a moderate mathematical level, is [Chihara and Hesterberg](http://www.amazon.com/Mathematical-Statistics-Resampling-Laura-Chihara/dp/1118029852/ref=sr_1_1?ie=UTF8&qid=1350820311&sr=8-1&keywords=hesterberg)
I wouldn't use a univariate time series method with 9 observations.
If you give more information about your data (and research question or hypotheses), someone might point out parametric alternatives (such as a GLMM). That you have non-normal residuals (the distribution of data is not relevant here) doesn't mean that you must use non-parametric methods.
@whuber What I meant by a partial correlation matrix is a matrix that has partial correlations in it (calculated for any two pairs of entries by partialling out all other pairs. In your words "a matrix of partial correlations". Yes, this is regarding time series as you have rightly pointed out. It is on the lines of back calculating a time series (168*12) if I have a pre-defined matrix having partial correlation data.
You do not find the significance level, $\\alpha$, you *choose* it, ostensibly so that it reflects your willingness to make a Type I error (i.e. to reject the null hypothesis when the null hypothesis is true). One the other hand the $p$-value measures how unlikely your test statistic, $t$, is to be observed assuming that the null hypothesis is true. If $p < \\alpha/2$ ($\\alpha/2$ because two-sided test), then you will reject the null hypothesis at the $\\alpha$ level of significance, and conclude that population mean weights in boys do not equal population mean weights in girls.
+1 for this clear and well-balanced response.
Is it sensible to add a randomized control group to be able to distinguish general improvement from improvement caused by the tool?
And the excellent efficiency of nonparametric and semiparametric methods is a reason not to pre-test for normality (besides the inadequate power of the normality test).
The Cauchy is (i) subsumed by the OP's mention of the $t$ (being $t_1$), and (ii) doesn't have a variance by which to normalize. So either it's outside the OP's conditions, or already covered in the OP's question.
+1  I made sure to read this book shortly after it came out.  I get plenty of opportunities to make statistical mistakes so I'm always grateful to have them pointed out *before* I make them!
@Gigili a polynomial transformation should do the trick (this would be similar to using a polynomial kernel). To see why, plot x versus x^2. You see that this polynomial of degree 2 already makes part of the subsets which were not linearly separable in 1D into linearly separable in 2D. If you move to a higher degree polynomial you will get full separability.
did you try GARCH?
Can you point to a crisp definition?
3) It's an interesting suggestion, but I don't know how to increase the swap memory in R. Could you give me some references? Finally, I had never heard of **RTextTools** before, but I found it's quite attractive to explore it. Thank you very very much again for your detailed comments. :-)
Well, what is the superfluous material? I think @Matt's answer covers the possibilities nicely. A results table is both conventional & handy, & there would be little sense here in not giving it fully if you give it - without necessarily expending a single word on discussion of an uninteresting effect.
Interesting points were made on a related question, [Effect of switching response and explanatory variable in simple linear regression](http://stats.stackexchange.com/q/20553/930).
That's what the original spreadsheet did, but it unfortunately doesn't come anywhere close to expected results.
If you have access to the [R](http://www.R-project.org/) statistical software, it would be worth taking a look at the [polycor](http://cran.r-project.org/web/packages/polycor/index.html) package.
You're the best! I'll leave the question open for a little while in case somebody has another implementation. For one, it seems that [binary indicators](http://stat-athens.aueb.gr/~jbn/papers/files/25_Lykou_Ntzoufras_2012_revised_final.pdf) can be used to impose variable inclusion / exclusion. This compensates for the fact that under Bayesian Lasso variable selection doesn't actually happen, since the betas with the double exponential prior will not have posteriors that are exactly zero.
Have you tried shiny? There are various approaches, which all boil down to running R from the server and fetching the results either directly with some form of integration or via files.
Thanks alot for the thourough answer and the suggestions!!
@Glen_b No, the $\\sqrt{1-r^2}$ form does seem to arise a lot, but in this context the correct denominator really is $\\sqrt{1-r}$. I originally pulled the formula from Cohen's (1988) textbook, but I have also verified it by deriving it myself (I think at one point I had the same suspicion as you).
What you are referring to is called an "individual participant" meta-analysis. Try starting with [this paper](http://www.bmj.com/content/340/bmj.c221.short).
This seems very much like homework.
Your terminology at the linked StackOverflow question was a little bit off, making the question confusing. If you want the confidence intervals of the *Pearson X^2 ("chi-square") statistic* for a given table, you might (???) be able to do this by Poisson-resampling the values in the table ...
Now knowing the name of the graph I was looking for, it looks its a sub-specialty of the Heatmaps called (quite aptly) "Calendar Heat Map".

If somebody (like me) is looking for an example in Javascript, checkout http://kamisama.github.io/cal-heatmap/
(+1) Welcome to CV, and be sure we welcome such questions. What do you mean by "This should probably use a *non-parametric* test" (emphasize is mine)?
[this book](http://books.google.com/books?id=ca5MkRbF3fYC&printsec=frontcover&dq=analysis+of+integrated+and+cointegrated+time+series+with+R&hl=fr&ei=QxB0TbPxMIGs4AaTxMmnDQ&sa=X&oi=book_result&ct=result&resnum=1&ved=0CCkQ6AEwAA#v=onepage&q&f=false) should answer all your questions.
Look into partial least squares, which is sort of like PCA but also tries to maximize the relationship with y.
Welcome to the site, @Prameela. This question seems to be *only* about how to deal w/ this error in R. As such, it may be off-topic for CV (see our [help page](http://stats.stackexchange.com/help)); but could be on-topic on [Stack Overflow](http://stackoverflow.com/). We will see if we can help resolve your issue here, if not we will migrate it for you (*please don't cross-post, though*). If it does go to SO, it will need a [reproducible example](http://stackoverflow.com/questions/5963269/how-to-make-a-great-r-reproducible-example) to be on-topic there; can you add a `dput()` of your data?
I think this is a duplicate, but I'm too lazy to search for it. Take $X\\sim N(0,1)$ and $Y=X^2$. $cov(X,Y)=EX^3=0$, but clearly the two variables are not independent.
I don't think the t-test is a natural candidate here, because your question as formulated is testing two observed means against two population means.  A t-test usually only tests between two sets of means.  Another alternative approach would be model testing under ANOVA... but it suffers from the same issue I've mentioned before... i.e. to 'see if the means are equal to the population' will essentially require that your endorse the null hypothesis, and does not yield a sensible question.  It should be clear that a chi-square does not provide an answer about the question, "does O=E".
Finding “the” significance is not always a fruitful approach. There might be a number of differences (difference in the overall/average blood level, more changes/variance for one group, different shape/profiles, etc.) Why did you take so many measures in the first place? You might want to think about how you expect blood level to change over time and what specific difference you are interested in. Otherwise, you might just as well compute the mean level for each participant/observation unit and do a test on that.
@Max In general, yes - but you need to make sure that that information isn't already contained in the regression.  E.g., if you add the channel to each visit, the regression model adjust its estimates to account for each channel's proportion of visits that result in purchases.  [I put together a little example](https://gist.github.com/3219819) to help clarify. Similarly, I don't think that adding a variable like "average tos of visits from this channel" would add any new information if tos and channel were already in the model.
The symmetry of the distribution around $0$ shows that you can reduce the calculation to expectations not involving absolute values: $y_b=c$ and $y_b=-c$ are equally likely when $|y_b|=c$ is given. If you intend $y_a$ and $y_b$ to be vector-valued, a similar approach ought to work (but will be more complicated to compute).
This is actually just a shortcut. In multivariate linear regression, the coefficients are proportional to the partial-correlation. So computing the regression, as Michael Bishop suggests, will give you a start. But the underlying idea is known as partial-correlation.
When you say you "check the output $C_t$...", do you mean the *filtered estimates* $\\hat C_{t|t}$, not the *true states $C_t$? What values for the two variances did you use?
The problem here is that I don't know the real underlying distribution since samples $X$ and $Y$ are just observations from a complex system.
Thanks for the reply, this makes sense to me. Could you suggest any time series methods?
To address dependence in time one can look at the relationship in first differences (changes) or returns. For example if they data is stock prices, it might be useful to look at the correlation coefficient between returns.
isn't this more of a comment than an answer?
Are you talking about traditional model comparison techniques, along the lines of AIC and BIC?
What makes you say that large sample sizes in hypothesis testing lead to false positives (i.e. type I errors)? Could you explain how you think one causes the other?
With neural networks you have to. Otherwise, you will immediately saturate the hidden units, then their gradients will be near zero and no learning will be possible. It's not about modelling (neural networks don't assume any distribution in the input data), but about numerical issues.
No. If you lived in Glasgow, Scotland, then most people who wear a "blue t-shirt", would probably be a [Rangers](http://en.wikipedia.org/wiki/Rangers_F.C.) supporter. You would be missing out on [Celtic](http://en.wikipedia.org/wiki/Celtic_F.C.) supporters. In Glasgow the [football](http://en.wikipedia.org/wiki/Soccer) team would be a proxy for religion.
It might depend on what you mean by "stronger".  At least two senses are natural in this context--statistical significance and size of effect. The latter is usually more meaningful, because the former is largely an artifact of sample size and data variation.  But how, then, are you measuring effect size when there is an *interaction*? It seems to me that at a minimum you will need to account for $b_1$ in your interpretation.
Thanks for your answer. I looked up block causality in Enders' "Applied Econometric Time Series". That does seem to be the appropriate way to look at this problem. I am still puzzled of how you would call the process $z$, however. Since it plays a very particular role in a Var of $x$, $y$ and $z$, I would assume that there is a specific term to refer to it.
According to *where* on the web? Those plots look fine to me. I would caution you however, to search for responses to some of the many questions on stepwise regression here (the issues are basically the same for any form of stepwise regression).
This should be a standard and easy question. Is it possible that no one is answering? Is maybe the question bad written?
This answer is quite specific of R (and its implementation of SVD PCA). Could give more details? What if, for example, a gene were to be found on more than one PC? What if the 2nd eigenvalue is very low compared to the first? etc.
Awesome, thanks very much.  Thats a great concise answer and makes total sense.
I still don't see how to do anything reasonable with fewer than many dozens of points, unless you can narrow it further.
How do you get rain on 0.5 days?  Do you have multiple 120 day periods, or are you counting the number of rainfall events per day?
This sounds like a Markov Chain problem where there are three states: `1.A lot of sales`, `2. Normal sales`and `3.Very little sales`. And you are asked: given the state was `2` at w=1, what is the probability of the system staying in `2` for 12 consecutive weeks? You need the transition matrix to solve this problem. 10% seems a red herring to me since if it is a constant amount, what you are interested in is actually the total sales amount (i.e. the system state).
Please read Wikipedia entry on Kolmogorov-Smirnov test to learn about "the discrete part". To my understanding KS is not applicable here.
How is your outcome variable distributed?  Try graphing outcome vs. predictor.  Compare the slope of the predictor for males and females.
@jbowman. I am doing time series prediction. I have used ARIMA, MA, ES, and random walk. I would like to know can i compare these time series based model to non-time series model like multiple linear regression ?
You should search around for "minimum detectable effects." A paper that I usually recommend is "Minimum Detectable Effects: A Simple Way to Report the Statistical Power of Experimental Designs" by Howard Bloom in *Evaluation Review* 19(5) (October 2005). My library proxy isn't working right now or I'd try to find a link for it. It's a pretty gentle introduction to the subject and offers formulas for calculating the necessary sample sizes.
@EleonoraLusito When people say shuffle data they are talking about permutation not bootstrapping. Do you mean permuting or sampling with replacement.  From your question I think you really do mean sampling with replacement since I don't see how order matters in producing your network.  But I don't understand what you mean by generating random networks. If you are simply trying to see how the results vary for a network producing algorithm under some null hypothesis about your data maybe I can be helpful although I don't know much about the literature on this. If that is not it, please clarify.
@jkd: This is a balanced design, so the df should match the df in the traditional least squares approach, which the K-R method does.
@Nick Stauner: I'd say that (a) you _should_ choose to increase your threshold when your sample size is small _and you can't collect more data_, because standard errors are invariably "large"; (b) you _should_ be more conservative about the null hypothesis when you have a gargantuan dataset, because standard errors are invariably "small" and t-ratios invariably "large" (see also Bruce Hansen, [Econometrics](http://www.ssc.wisc.edu/~bhansen/econometrics/Econometrics.pdf), §8.8). However, I'm going to add something to my answer.
Take a look at http://en.wikipedia.org/wiki/F-test_of_equality_of_variances (the SD is the square root of the variance)
(ctd) ... the area is quite large; if you narrow it down a bit I can probably find you some particular applications.
It might help to know more about the format of the data here. What is the criterion for assigning tags (semantic information, word identification, word type counts...)? How many tag categories are there in total (does 5 refer to the maximum number of joint classes here and how is it determined)? What type of regression did you try that leads to a classification of data?
+1 This is another way of modeling parametrically the baseline hazard http://data.princeton.edu/wws509/notes/c7s4.html
Don't scold, Michael (as you know, personal attacks are inappropriate on this site, as they should be everywhere): you brought up the 45 degree issue and that clearly is misleading, despite the disclaimer at the end. I urge you to edit your reply to prevent the confusion.
Do I understand correctly that you want the confidence interval for the prediction of _one_ case?
A fuller description of the data available to you might inspire effective replies.  One thing I wonder is whether you have any information about whether a user found the character they were looking for or eventually abandoned the search.  Perhaps even time stamps could help.  Another point: because "correlation" has a specific meaning in statistics which differs from the sense used here, it could mislead some readers.  You might consider using an alternative terms such as "association" to describe what you are attempting to measure.
@GaëlLaurans Yes, 0.1 comes from the "distance" between the percentage found (15%) and any boundary of the confidence interval. But my calculations are certainly wrong !  I'll add this precision in my post.
You may also find the following paper helpful: "Structural equation modeling in practice: A review and recommended two-step approach" (Anderson & Gerbin, 1988). While it doesn't refer to WLSMV, it contains discussion on various estimators.
http://ecd.ralmond.net/ecdwiki/ACED/ACED - this actually looks like a promising set of data (Q-Matrix is defined, about 300 students).
I agree that there are situations where the main effects *can* be interpreted -- Venables takes a very strong line -- but there are lots of situations where they are difficult.  I think "don't do this unless you know what you're doing" is a reasonable default setting ...
It is not appropriate to use variable selection methods that do not incorporate shrinkage.  Lasso or quadratic penalty (or mixed: elastic net) should work OK.  P-values from ordinary stepwise methods are not valid, and using measures of association to guide model selection is frought with difficulties.  Note that variable selection guided by associations with Y does not help in any way with the "too many variables too few events" problem.
Thanks Marc, this is exactly what I was looking for.
and [here](http://cran.r-project.org/doc/contrib/Faraway-PRA.pdf) (from page 72 on), [here](http://www.stat.berkeley.edu/classes/s133/Lr0.html).
Sorry for not answering this earlier. I got an answer elsewhere indicating how this can happen. Basically the tests reported by default by Anova(), are formulated assuming that terms to which a particular term is marginal are absent (i.e., have coefficients that are 0) which can lead to discrepancies with the estimated significance given by summary().
No problem. Taxonomy is the hierachical structure of an ontology. It implies a child-parent relation. Like: a Lion is a mammal. A mammal is an animal. An ontology can have more relations than the "is a" s.
@Skliwz, I'm not sure how you would normalize a multiple choice (ordinal) answer.  Also, remember that linear transformations of continuous predictor variables are sometimes desirable for reasons discussed here: http://stats.stackexchange.com/q/7112/3748 and here: http://stats.stackexchange.com/q/19216/3748 but they won't change the models predictions barring some unusual computational issues.  If everyone enjoys the outdoors, the outdoors equally the outdoor answer is less relevant, but I don't think its really a problem for the model as I specified it.  (Not that my model is perfect)
@Jeremy: Thanks for the great editing job !
Ah, I think I understand one point you are making: if I fit the model for complete cases prediction using imputation then that will improve the complete case forecast, vs fitting it with just the compete cases. The remaining question is what is best for the incomplete cases?
If you insist on viewing $S$ in your equations as a measurable real-valued function on an abstract probability space--which is correct but problematic for what you are trying to do--then you will run into even more trouble making sense of its derivative with respect to $\\sigma$. (Recall that derivatives are limits of difference quotients: what quotient do you have in mind and how will you define the limit?) I am suggesting likelihood *is* a useful approach due to the intimate connection between the MLE (with Gaussian residuals) and the OLS solution.
Does that mean if I Box-Cox transform both x and y, then $\\lambda$ from each of those transforms counts as a parameter too?
Could you recommend some literature? Especially regarding the implementation of mixed-models in R?