tttjjj/elt_bench huggingface dataset:
The noncentral t is not a location family.  It is [usually defined as a ratio of two random variables](http://en.wikipedia.org/wiki/Noncentral_t-distribution) and only the numerator is shifted by $\\mu$.  That's why the entropy depends on $\\mu$. It would be a good idea to plot a few PDFs of this family so you can improve your understanding of it.
(1) What you seem to be missing is that when you change $K$, you change $LL$.  (2) What do you mean by "MDL"?
Just noticed I didn't specify those were `R` packages.  I know at least SAS has comparable functionality in `Proc Quantreg`.  You can also use the `gbm` package in R and a quantile loss function to get similar answers.
For third question, you want to know a R method to find out estimates or statistical equation?
@robo, I am somewhat familiar with Gretl, as far as I remember it is certainly possible to specify constant. Which menu option are you choosing? Can you include the output of gretl model in your question?
(1) I up voted Bruno's comments. seemed sensible (2) estimates on split sample are unstable unless sample size very large. So it is standard in fields that have very large data sets, but not in those that don't. (n>20k? what large is isn't clear). And if your sample is so large, getting hard to over-fit model with logistic regression (apparent performance=oob performance).  (3) For k-fold validation, 2-fold with give two estimates so not quite split sample. Choosing K has been discussed here (http://stats.stackexchange.com/questions/27730/choice-of-k-in-k-fold-cross-validation).
@Glen_b: Thanks! Sorry I do not have enough time to write a good enough answer.
Can you think of a counterexample? (**Hint**: Start with $N=2$ and Bernoulli random variables.)
I'm also a big fan of Kendall's tau. Pearson is far too sensitive to influential points/outliers for my taste, and while Spearman doesn't suffer from this problem, I personally find Kendall easier to understand, interpret and explain than Spearman. Of course, your mileage may vary.
@Milk It's not that simple.  Suppose you invest 100 dollars. After year 1, you take 25 out, leaving 100.  After year 2, you have lost 40, so you put the $25 back in *plus* another 15.  After year 3, you have 125.  *In toto* you were forced to have 115 in cash (**not** 100), on which you made only 10 after 3 years.  All this assumes you cannot earn interest on the unused cash sitting around and you cannot borrow money.  This only begins to hint at the mistakes that will be made when using oversimplified formulas for expectation.  Reading a good text on financial calculus will serve you well.
Note that in actual computations, one should not be forming $A^T A$ directly; exploit the QR or singular value decomposition of $A$ for this.
I agree with this nice answer by Marc.
@gakera: You got me wrong. If you don't understand what you're doing, you should definitely try to learn it **before using it**. That means, reading up on statistics and following a course. So, a good model should include the variables that are formulated in the hypothesis. If you base yourself on the "***" variables, you need a thorough course on modeling first. You obviously didn't understand my last comment. Sorry for the direct communication, comes with the guy. Nothing personal.
There are many ways to approach this. For one thing, you shouldn't expect any answer here to be *the* answer, nor for it to cover all the possibilities. With a question like the one you've posed, you could be about to embark on quite the voyage of discovery, if you're willing to put in the time and the effort. I'd start by reading introductory pieces and whatever more advanced books/articles you connect with, on regression, path analysis, and structural equation modeling. I especially recommend James Davis' short, $18 book, *The Logic of Causal Order*. Good luck!
@user31264, I added some R code in my example to demonstrate my suggested approaches.
$e=b3X3+u$, so $cov(X2,e)=cov(X2,b3X3+u)$=$b3cov(X2,X3)$ which is not equal to zero, as long as $b3$ is not equal to zero. Proved.
If you want help specifically with SPSS code, your question would be off-topic here. There is an interesting underlying statistical question here, but that means the question should be thrown open to people regardless of what software they use. As it stands, your question seems more the first, rather than the second, so I suggest that you either broaden it or withdraw it and seek help in an SPSS forum. (Stack Overflow is not really for this kind of question, unless you show code and give a reproducible example.)
I raised a question [here](http://stats.stackexchange.com/questions/83186/can-you-do-statistics-with-3-data-points) because I really want to know if this a correct approach from a statistical point of view.
Could you please tell us what is being sensed and the meaning of the sensor values? Otherwise, we can only guess what the relationship between the two sets of responses is supposed to be, which makes it difficult to suggest any meaningful or useful procedure.
"it is convenient to let the sample space consist of all possible names (rather than all names that we know of)" - how is that convenient?
@user3727: +1 I am not sure I am doing this right but when I do `resid(my_model)` on my second one I am getting a NULL. At this point, I am not sure I should be considering the previous attempt because the experiment design was pretty careless on my part.
Just to be clear, it seems you want a CI for $p_1/p_2$ where the *estimate* $\\hat{p}_1$ is $0$. Because you did an exact significance test, you should be able to use essentially the same calculations to obtain an exact confidence interval.  What formula, then, did you use?
Because it caused some confusion in one of the answers, please note that the assertion that the "CI width should approach 1" is either meaningless (1 what? What are the units of measurement?) or just plain wrong.
Kurtosis by definition is invariant under affine linear transformations, which includes standardization. That fit is pretty bad, by the way: using a histogram-like presentation only obscures the differences.
A and C don't have the same value. If you mean 'don't the entire expressions have the same value' then trivially they do, since they derive from decomposing the same event (ABC) in two different ways (which is nothing more than exploiting the symmetry of symbols in ABC -- the event ABC is the event CBA), but that doesn't make the expressions themselves "the same" -- because what is conditioned on in each expression is different. If you intended to only refer to the same value, you could edit your wording to more clearly reflect that.
Historically, the very first demonstration of the t-test (in "Student"'s 1908 paper) was in an application to sample sizes of size *four*. Indeed, obtaining improved results for *small* samples is the test's claim to fame: once the sample size reaches 40 or so, the t-test is not substantially different from the z-tests researchers had been applying throughout the 19th century. You may share a modern version of this paper with the panel member: http://www.york.ac.uk/depts/maths/histstat/student.pdf.  Point out the investigation in Section VI, pp 14-18.
Please don't re-post questions. Asking once is enough: http://stats.stackexchange.com/questions/85868/using-knn-for-2-dimentional-data
Are conditions different diseases or symptoms (outcomes), or are they explanatory variables that interact with treatment (like smoking would be in a study of alcohol consumption and cirrhosis)?
Sorry, yes, I kind of understood that. I should have been clearer in the question. What I kind of mean is what is the real world statistical/conceptual effect of that mathematical requirement? Does it make logical sense to negatively weight data that you know isn't giving "negative" information (ie. data that isn't lying)? +1 though, answer is still useful :) clarified the question.
@LCialdella: I just read about square of pairs, it in my case making RMSE lower but slightly, i think if representation will be correct it will be work better, I try to capture what pair of features predicting good when used combine, some of feature pairs are highly correlated, i think that regression can learn to score more feature pairs that are not strongly correlated but i need function that will be have correct representation, when 2 highs are combined that should be higher, when 2 low values then should be lower not higher
... it would be just a lot harder to explain to my boss ;) a nice clean equation for calculating the CI would have been better.
Hi Macro, thanks so much for your help! I have upvoted your answer and yes, I am considering eventually accepting it... Thanks again for your help! And I have followed your suggestion and started a new discussion ...
The SE of the mean is determined by the variance of the sample average, which is directly related to the variance of the underlying distribution.  To understand this better you might want to research our site for threads concerning standard errors.
Your method's right, you're just putting the number in the wrong box. The center of the convolution is at (0,0) in the original image, but not in the output, because the center of the convolution could be located outside the bounds of the original image.

The center of the convolution could be at the point (-1,-1) on the image you've drawn--do you follow that?

It may be helpful for you to think of the top left-hand point in the output of the convolution as (-1,-1) instead of (0,0)
Casey: This is time series data thus the observations are correlated thus you can not use the ANOVA approach as you don't have independent observations
Why not just describe the curves with some summary statistics instead of formulas?  Would, for instance, saying that $y_2$ is approximately a Beta$(2,2)$ scaled by $10$ be more illuminating than saying it is symmetric around $5$ with a range of $10$?  In fact, your data are so scarce that any formula is going to appear far more accurate than it really is.  Using formulas seems to introduce a certain unnecessarily arbitrariness to the description of the data.
I have nothing **statistical** to add to the excellent answers given (esp. the one by @whuber) but I think the right **answer** is "R-squared: Useful **and** dangerous". Like pretty much any statistic.
CI should not be .95, it should be a z score for a given level (e.g. 1.96). What you have labeled "Resp" is usually called p.
@user1516425 a "flat prior" refers to a uniform prior, usually, though the connotation to me is that you are allowing the prior to be improper.
@Mike Lawrence I tried using your ezPredict to see if it could help me understand what's going on, but it wouldn't work on my lmer object -I think because my dichotomous predictor variables are not coded as factors. You seem to have a pretty good handle on these LMMs. Can you explain to me -or point me somewhere that can- why the intercept in my RT model is not the same as my observed grand mean?
(1) One advantage of neural net approaches to survival analysis is that they do not rely on the assumptions that underlie Cox analysis. You can get by with simpler Kaplan-Meier estimates for censored cases, and avoid this complexity. Look at [link](https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2813661/) and the references therein for what seems to be something close to what you have in mind. (2) Make sure you have a good handle on the meaning and reliability of the underlying clinical data. Staging and other clinical variables can have different meanings among different types of cancer.
Good links here: http://www.r-statistics.com/2009/10/free-statistics-e-books-for-download/
Something is the matter with your application of the S-W test: because the values of $C$ must be one of the three elements $\\{-1,0,1\\},$ this test is certain to find that the distribution is non-normal.  The test, however, is irrelevant, because the underlying distribution of $C$ does not matter; only the sampling distribution of the *mean* of $C$ matters, and that will be very close to Normal except in extreme cases.
Are you opposed to using a full versus reduced test instead?  I think it would probably make more sense to fit a reduced model where you only have a single variance component and then use a likelihood ratio test to compare the full and reduced models.
The answers to this question will vary, as they should, because both "data-generation process" and "model" are used in varied ways by various authors.  @Weijie, do you have a particular reference in mind?
Thanks Glen. I was hoping there'd be something analogous to central limit to argue in favor of gamma distributions, but I guess not.
Just for clarification: Nunnally/Bernstein suggest to treat a variable as continuous if it has at least 11 distinct values (p. 115). Where is that "12 points imply interval scale" rule of thumb from?
Thanks. I have tried and I get         

Overall

reghght 11.710091 / regfreq  7.432101 / regtem2 6.160171
/ regsreh  2.325263 
In which unit are these values ?
I would look into partial least squares for this type of problem. I wrote about [PLS](http://www.statisticalanalysisconsulting.com/partial-least-squares/) on my blog
You need to understand "results" in terms of the estimated relationship between the independent and dependent variables.  *Standardizing will not change this.* (In fact, almost all regression software will automatically standardize the variables in order to do its calculations.) Because you changed *how you express the variables' values,* of course the estimated coefficients will change! Multiplying a variable by a constant merely changes the units by which you measure it. There's absolutely nothing wrong with that.
Whatever approach I end up using, Monte Carlo will certainly be used to test the implementation.
What do you mean under "hierarchical regression"? And please provide the two commands in syntax which you are comparing.
Many thanks.  I *do* use R extensively, but will also check out the book.
But doesn't the third (central) moment do this in a more stable and practical way?
Please see my comment on Dikran's answer.
By $z$-scores, should we assume you meant $(x-\\bar x)/s_x$? You should not expect people to be familiar with specific instruments, so you could just as well called them X1, X2, X3, etc. A link to the blog would've helped to see through clearly what the recommendations were. Finally, different disciplines have different ideas about "large" and "small" samples -- do you have 10, do you have 100?
When the joint and marginal distributions have been determined, why is it necessary to compute $H(X)$, $H(Y)$ and $H(X,Y)$ and use the formula you exhibit? Can't the mutual information be determined directly via the formula given by the OP since everything needed for "plugging in", viz. $p(x,y), p(x)$ and $p(y)$ are known at this point?
Quasi-Poisson accounts for overdispersed data (the type of data that arises from slight unmeasured correlation between observational groups. You'd need a justification to motivate using quasi). Let's table that idea for now. Your syntax suggests you're using R. Import the package `lmtest`. You can perform the likelihood ratio test of the null hypothesis that all regimes have equal abundance by typing `lrtest(glm(abund ~ regime, family=poisson))`.
By the way, I just noticed what seems to be a printing error on Colgaton's book: he defines $p_c = \\sum_{i=1}^k p_{i+} p_{+j}$, but this $j$ comes from nowhere. I suppose it was meant to be $p_c = \\sum_{i=1}^k p_{i+} p_{+i}$, otherwise I am not sure it makes much sense.
Is this a serious answer or a shotgun approach where if you mention enough smoothing methods you will hit one that applies.  Maybe they all could given the vagueness of the problem. The idea of smoothing is in general is to separate signal from noise and all filtering approaches attempt that.
Please see the forum's policy on homework or similar questions. You are asked to show your reasoning in a draft answer and ask for comments. Just giving the question is not enough.
Related question: [Can principal component analysis be applied to datasets containing a mix of continuous and categorical variables?](http://stats.stackexchange.com/q/5774/930)
Sorry I had misinterpreted that part of your note.  The mean absolute error is reasonable and has much written about it in this context.  It is almost the same as the ordinary $R^2$ because of special properties of squaring things that are Y=0 and Y=1.  See Buyse, M. $R^2$: A useful measure of model performance when predicting a dichotomous outcome Statistics in Medicine, 2000, 19, 271-274 and Hu, B.; Palta, M. & Shao, J. Properties of $R^2$ statistics for logistic regression Statistics in Medicine, 2006, 25, 1383-1395
Many a thanks for the edit.
I agree w/ @PeterFlom, someone is categorizing a continuous variable; *blood pressure* (a continuous variable) has been turned into "High, medium, & low" (discrete categories). However, we can also ignore these details if you're just wondering about how to analyze repeated discrete measures in general. The answer requires specifying how you're thinking about the problem. I discuss those issues here: [difference between generalized linear models & generalized linear mixed models](http://stats.stackexchange.com/questions/32419//32421#32421). More info exists at other threads linked there.
Such an approach might instead be picking up *variance* misspecification, perhaps the variance changes according to some covariates that are in the OLS model for the mean.
Usually such constrains occur when you solve some linear programming problem, thus you have the gradient of the objective function to maximize (or minimize). For an invertible $A$ in $Ax = b$ the Karl's point goes the solution is unique for the system of linear equations with $n$ unknowns and $n$ equations. Probably http://ieeexplore.ieee.org/Xplore/login.jsp?url=http%3A%2F%2Fieeexplore.ieee.org%2Fiel5%2F4531369%2F4537177%2F04537325.pdf%3Farnumber%3D4537325&authDecision=-203 could help.
Thanks for your thoughts Glen. Seems like I have my work cut out for me.
Does it have to be a paper? This is a standard result found in *many* text books. I'd even suggest providing the proof yourself and placing it in an appendix.
Rather than guessing (and attacking the question), please just ask the OP for clarification, @Michael.
Nice answer!$\\phantom{.}$
I really don't see how "relationship" implies such a distinction as they made. I'd be pretty content with the conclusion in your final sentence.
@Roland Thanks for Reply. No, I'm looking for the way to exact y values of whole regression function when x is cutoff.
A setting where you are confident that the regression coefficients represent the effect of the regressor on the regressand.
What does MOOCs stand for?
Thanks for your answer, but what I am really interested in knowing is under what cases using medians rather than means provides better estimates the central tendency (median or mean) of the function.
Thanks, code worked like a charm. 

However, I realised I have a theoretical problem. See, the data is about how many observations of bicycles and other vehicles were observed in an hour in an area. There are 10 such one-hour observation periods for areas with bike roads, 8 for those that don't. 

I thought I would sum those 10 (8) observations together to get the required data for two binomial distributions (x,n,y,m). Is this a valid technique, assuming these 1-hour observation periods are i.i.d? 

What other models could be used in such a case? I don't see how Poisson could fit here.
Short answer: yes. Longer answer to follow.
@IrishStat, what I had in mind for this example, is dividing the data into train and test part, fitting different ARMA models for the train set and using the forecasting performance on the test set to select the "best" model. I do not claim that this will be the **best** model, only best to this criteria.
Can you  tell us about your problem succinctly? I'm reluctant to go and read a paper to find out what it is. Perhaps an example of some data which look like yours?
Just that if if you measure $A$ as say 50%, & $B$ as say 30%, then you *know* $C$ is 20% because the total must be 100%. So the last principal component will always have an eigenvalue of zero: you might as well just carry out PCA on A & B; & even then linear correlations are probably not so relevant to you.
I basically have 80 species and 16 samplers (times three changes= 48 samplers). I had 4 sites with 4 samplers in each (but each pair of samplers in each site were sampling the same media) so hereth lie the problem. Excellent re: aggregating the data, as my data is overdispersed then a negative binomial glm will suffice then yes?
A bootstrap ain't gonna work here. It won't work with the data sets in which pretty much every observation is correlated with pretty much any other observation; it needs independence, or at least approximate independence (mixing conditions in time series, say) to produce justifiable replicates of the data. Of course there are special bootstrap schemes, like the wild bootstrap, that may circumvent these issues. But I won't bet much on this. And you really need to look at multivariate statistics books, and follow them, so as not to obtain yet another indefensible hockey stick as an answer.
@oisyutat yes I assume that the sensitivity and specificity are not affected by patient selection, which may not be legitimate, eg if the test has only ever been used on people from a certain age group. Incidentally I don't believe there is really a 'probability' that the patient has the disease, it's a case of what you belief on the basis of evidence you've accumulated.
You may find the discussion in [When should you center your data & when should you standardize?](http://stats.stackexchange.com/questions/29781/when-should-you-center-your-data-when-should-you-standardize) useful.
Oh, so you are the villain, then. This one has a single categorical predictor, so it is an ANOVA model to me, as I said earlier. If you had information at different levels (e.g., state \\ school \\ students, with data on states, on school, and on students), that would sound more like a mixed model to me. Basically, if you can do this as sums of squares, that's ANOVA; if you can do this as a regression model, that's a regression model. If doing the maximum likelihood/REML is absolutely unavoidable (as it is in binary response case), that's a mixed model alright to me.
@Glen_b: isn't your comment already an answer?
Can you please try to make it more clear?
Also, you can factor out a constant from the covariance in this step:

$$
\\frac{1}{n} \\frac{ 1 }{ \\sum_{i = 1}^n(x_i - \\bar{x})^2 }
    {\\rm Cov} \\left\\{ \\sum_{i = 1}^n Y_i, \\sum_{j = 1}^n(x_j - \\bar{x})Y_j \\right\\}
$$

even though it's not in both elements because the formula for covariance is multiplicative, right?
I sincerely appreciate your response.  However, the purpose of StackExchange is to provide answers, not references.  Statisticians seem particularly prone to this reference problem.  Is there any detail you can add about how to use residuals.lrm?  For example, an example command and an example of interpreting the graph for the house.plr example?
This is incorrect because you are following a different convention for subscripting the $X$'s: they start at $0$, not $1$.  Bear in mind, too, that the convention with Markov chains is to *right* multiply by the transition matrix.  You can always tell which convention is in use by checking the sum-to-unity condition. By multiplying on the left by this $P$, whose *rows* sum to unity, you are not computing correctly.
I made a little drawing of what I'm computing: http://img109.imageshack.us/i/mfpa.jpg/ I hope that is of help to you. A(t) is some  observed quantity. $A_k$ is an upper threshold and $A_0$ some fixed lower one ($0 \\not\\in k$). In physics we call the $y_{ki}$ first passage times. So I would say that there are high correlations for $k \\neq l$, probably the correlations for $i \\neq j$ are not so important. So computing this $cov(y_{ki},y_{kl})$ is in fact the problem. And thanks for the insight in stationarity.
Closely related: http://stats.stackexchange.com/questions/421/what-book-would-you-recommend-for-non-statistician-scientists