tttjjj/elt_bench huggingface dataset:
This is a question of statistical power, & determining it is the subject of a power analysis. If you were to specify the power that you want to be able to achieve, & the N that you will have, the minimum detectible accuracy can be determined. W/o this info, this question cannot be answered.
(1) hard not to point out that AIC based criteria are intended to produce the result you're not happy and favored over a p-value cut-off that will produce an even worse model (2) you might get a better answer if you expand your question to mention which part of the three classical methods you've considered (a) variable reduction without looking at outcome (PCA, redundant variables etc) (b) shrinkage methods (elastic net) (c) cross-validation methods (deletion substitution addition) and (3) if you somehow show awareness regarding the mirage issue mentioned above.
Yeah, this is probably an error in the problem statement. The sum of probabilities on edges going out of every state must be one. Also having two edges from state 2 to 4 makes no sense, so probably one of them should be reversed, and then everything will be ok.
This helps a lot. I think where I was going wrong was confusing 'check if H0 is correct' and 'assume H0 is correct, then see if the sample fits'. At least, that's how I think I'm supposed to interpret it...
$R^2$ is another excellent measure, but if calculated the usual way allows for a linear recalibration of the predictions so does not penalize for predictions being off by a constant or a constant multiple.
Thanks a lot, everyone!I now understand better the problem:) I meant to treat this rating as ordinal, which is discrete...but 2.5, 4.5 actually made it continuous in a sense, yeah, it is conflicting.

So still this should remain ordinal and discrete,so I think mapping it into a 9-point scale can be a good solution.

1   -> 1
1.5 -> 2
2   -> 3
2.5 -> 4
3   -> 5
3.5 -> 6
4   -> 7
4.5 -> 8
5   -> 9
I'm definitely open to other ways of reducing/visualizing the data. Scatter plot was just the first "see what I'm dealing with" attempt, but I am curious how to manage data with a distribution like this on scatter plots. Transparency is a good idea.
I tweaked your $\\LaTeX$ to get it to render. Please ensure it still says what you want it to.
A0B0C0 is a control state of the G. Every experiment starts from there and it’s finite. It not the same wall all the time. Workflow is following. Wall containing B3 treated with A1 and without C (C0) - G is measured(in 20 different places). Wall discarded. Next Wall B3 treated with A1 and C1 and another 20 G is measured, than B3A1C2, than B3A2C0; B3A2C1; B3A2C2... and so on. In a separate experiment we decided to have a look at effect of B1, B2, B3 on the level of cement in the mortar via doing HPLC and quantifying the level of cement (nmol/mg of protein).
There is an error in your first "equality". Do you see why?
How could you possibly get values *above* 10 in your histogram when you're trying to truncate the distribution at 10?
and, depending on how those 15 people were selected you might only be able to generalize about those 15 people. How you sample is selected is critical to the ability to generalize to a larger population, the decisions and recommendations you can make.
Normally one does not set out in a data analysis to compute a correlation coefficient: one has specific analytical objectives, such as to identify a relationship among variables, or make a prediction, or estimate something. Computing a correlation coefficient is just a technique used to get you there. Whether it should be computed at all, and if so how it should be computed, are questions best decided within that larger context. Would you mind, then, editing your question to share some information about your objectives and what you hope to achieve with this calculation?
I mean, you'll need a swath of theory and applied books to have the equivalent of a statistical training, I own ~30 and am hopelessly underprepared in several areas. Still more depends on your desired area of application: Economics? Biostatistics? Engineering? I recommend you go consult the required texts from a credible statistics program from a university you enjoy.
The command `density` implements, by default, a Gaussian kernel with bandwidth given by `bw.nrd0`. You can then easily write this function: `kg = function(x) return(mean(dnorm((x-hx5)/bw.nrd0(hx5)))/bw.nrd0(hx5))`. Therefore you can evaluate the density at the desired points using *e.g.* `kg(median(hx5))`.
I'm afraid the data is not accessible at this moment - still going through IRB process to get access to it.
I'll say that JMP is quite good.  Even if one is very proficient in R, there are times when JMP is a faster way to navigate and analyze data.
Gavin, you don't need to remove *any* of your data. Since your purpose is the regression, just use the results of robust regression. If you're instead using this to detect outliers, then some judgment is called for: an "outlier" could be any value with a weight substantially less than $1$. But how much less? You will need to decide that yourself by studying the relationships between the weights and other information you might have about data quality.  (The last `R` code example in this answer shows how to extract values of one array by thresholding the values in another parallel array.)
@niandra82  thank you for your comment. I created a new topic and could you please explain that? http://stats.stackexchange.com/questions/108590/how-does-cross-validation-and-bayesian-method-overcome-overfitting-problem
(+1) About R packages, there're also [glmnet](http://cran.r-project.org/web/packages/glmnet/index.html) (implementation with coordinate descent algo., Friedman and coll.) and [penalized](http://cran.r-project.org/web/packages/penalized/index.html) (allows to keep some var. unpenalized). Of note, F. Harrell provides penalized ML estimation for GLMs (see `lrm`, or his RMS textbook for further info).
As much as everybody would like it, there is no such thing as a threshold for this, so I cannot make statements on whether 0.01 will work: you need (the relevant) multiple testing correction for this, which will depend on your design. Even then: I eschew using the results from a single GWAS without individual confirmation.
omg! a paper which actually writes down the likelihood function for the observed data!
RRDTool can be found here: http://www.mrtg.org/rrdtool/index.en.html
Your question lacks a host of critical details. The situation you just programmed gives no context as to what you are trying to accomplish, nor to what "previous non missing value" refers to. Is your data a time series?
The difference between significance and non-significance is not necessarily significant. Are you confusing small changes in p-values across the .05 border with practical importance? or has he size of the main effect changed in a theoretically meaningful way?
As said in your answer, primarily LDA is used to do dimension reduction, but if the purpose is just classification, then we can just simply use the Bayes approach, right? But if the purpose is dimension reduction, then we have to take the Fisher's approach to find those directions on which we will project the original input $X$, right?
@Brett: Fantastic!
@Henrik, nothing wrong with your answer, I will accept if no other answers are presented.
@Elvis Neither do I understand. A confidence interval based on the profile likelihood should be constructed with the help of the $\\chi^2$ percentiles, which appear nowhere.
but the answer is: this is not a sufficient statistic.  Also, joint PMF gives $T=x_1+x_2$, and I see why that is true.
@Michael Because *their* answers are also marked correct! (Haven't you ever taken a multiple choice test in which more than one answer was correct? Did that bother anybody?)
There is no way to sample uniformly if there is no upper bound.
"*Will ... serial correlation be introduced as a result of having 
both AR terms and the time term?*" -- clearly, since AR is 
already serially correlated; the time trend introduces an additional source of
'dependence'. Often this sort of thing can be 
identified, but there can be a number of issues. A couple of relevant papers - [no.1](http://www.econ.uzh.ch/faculty/wolf/publications/etcaSub.pdf) [no.2](http://people.bu.edu/perron/papers/robust-testing.pdf)
Are you sure that x1 and x2 are coded as factors? Should `pre` be a factor?
Stevens' typology has been debated and disputed since it was first published. It's a sometimes-helpful framework, not a theorem.
As a user of TraMineR, I am wondering when this feature will be available.
Thanks for your answer. This is something that interests me, because some text books say that the Tukey HSD comparisons should not be done if the overall test is not significant. I've done a few simulations which showed that the Tukey HSD confidence intervals have the correct coverage regardless of significance of the overall test, so I don't quite understand the recommendation to only use them if the overall test is significant. I'll be interested to hear some other opinions.
@cardinal you are right. I fixed it.
@SabreWolfy I wasn't sure what OR you are referring to: originally, I thought you meant the OR from the classification table that compares actual category membership with predicted membership (the `cTab` part in my answer). But now I see you probably just mean the $exp(b_{j})$: like fabians explained, $exp(b_{j})$ equals the factor by which the predicted odds change when $X_{j}$ increases by 1 unit. This is the ratio of the odds "after the 1-unit-increase" to "before the 1-unit-increase". So yes, you can just use fabians answer.
This is quite interesting: http://www.stat.washington.edu/peter/342/nuzzo.pdf
What command are you using? `regress` automatically displays a table with the sums of squares.
@PeterFlom Exactly, that was the model.  Each block contains one quantitative variable. variable 1 = standarized residuals of linear regression for each individual. those residuals resulted from predicting pre-test-values out of post-test-values (this was done to get a value for the individual learning potential), variable 2 = lobal score of a working skill assessmentstandarized residuals of linear regression for each individual. those residuals resulted from predicting pre-test-values out of post-test-values (this was done to get a value for the individual learning potential)
I'd suggest considering a [logspline](http://cran.r-project.org/web/packages/logspline/) density estimate
Just to illustrate @Hong Ooi's excellent answer: Try to transform the log-odds back to probability: $\\exp{(-16.8758)}/(1+\\exp{(-16.8758)})\\approx 0$.
@fg nu, can you try replying to the above question?Thanks
@Cardinal: done.
Try subtracting 2.5 instead of 3 from $n$ :-).
(+1) I often prefer to sketch out a graph on paper first to reduce the chance that my design decisions will be guided by the path of least resistance created by the graphing software.
(+1) Note that that the constraint is one of maximal covariance, not correlation, in case of PLS regression.
`bsline ..., p(1) ... ` is a nice shortcut:  See http://data.princeton.edu/eco572/smoothing2.html
@Sriam Yes I was saying that, but when thinking about it more I think converting the data into a probability distribution might be a bit weird in this instance...not sure...
I've edited my original question to ask for advice on why `ladder` returns missing values only.
@Allen you should add that to your question (now I've done it for you)
$\\epsilon_t$ is not iid though
"Counterfactual" is often used to denote a specific philosophical perspective on causality and relatedly in causal estimation, so your use of the word in your question is a bit puzzling. See for example, Maldonado, G. and Greenland, S. (2002). Estimating causal effects. *International Journal of Epidemiology*, 31(2):422–438.
I don't think this is correct.  If sparsity is achieved via regularisation e.g. a LASSO approach, then the set of feature subsets that can be generated is likely to be much smaller than the number investigated by an exhaustive search.  This means there is less opportunity for over-fitting as the "model space" is more highly constrained.  I wouldn't recommend exhaustive search unless the data set is *very* large and the number of features is very small (of course it quickly becomes computationally infeasible with the number of features anyway).
"*would like to test some plausible hypotheses*" -- it sounds from that phrasing as if the hypotheses were generated based on the appearance of the data. If that's the case, you have the problem that you can't readily calculate appropriate p-values if you try to apply the test to the same data that generated the hypotheses. The distribution of the usual test statistics would be affected by looking at the data to generate the hypotheses.
No, only 7 for race and 19 for state. 26 degrees of freedom is often (but not always) not a problem. The large number of parematers you report would be the result of an interaction between race and state.
I just found the second paragraph confusing, that's all, because (1) putting "correct" in quotation marks in the first paragraph suggests you don't really believe what you wrote, so then (2) following that up with a discussion of the chi-squared test makes it look like you really are recommending it as "most correct." As such, your answer prompted my initial comment looking to clear up that confusion.
(+1) very good ideas, but I especially like the point that data analysis should be connected to a (may be  roughly) predetermined goal.
$\\delta t$ refers to the delta (or relative) time between points. So in your case, between the sample at midnight and 1am, there is a $\\delta 1 = 1$ hour. 

From your description, it sounds as if what you're mostly interested in is total library usage, which can be represented (in a daily sense) as either the **sum** of all head counts or the **mean** of all head counts. If you convert your data from 7 pts/day into 1 pt/day and plot each Semester individually, you *may* then be able to draw some conclusions about your samples.
Hi and welcome to the site! Could you clarify what the numbers from the papers are (in the example)? What are the variables and what is meant by "coefficient"? Also, what do the numbers within the parentheses denote? Thank you.
@Zach Thanks for the link. As for the second comment, I was actually hired to work on this project (I'm a university co-op student), so I have ample time to work on this problem. As for the dataset, I am not allowed to share it, but I may be able to link to some articles that are publicly available concerning the type of analysis we do on the data if you want to read about it.
The differences seem relatively small, though probably too big to be simple numerical issues. Could it be the difference between normalizing by $n$ or $n-1$, for example, when calculating an estimate of the covariance prior to computing the SVD or eigenvalues decomposition?
You need to be more specific. The meaning of $
\\frac{\\partial \\mathbf \\Sigma(\\mathbf \\theta)}{\\partial \\mathbf \\theta_k} \\overset{p}{\\to} \\mathbf{0}
$ is not clear. You have a matrix-valued random function, you want to it to converge in probability to the zero matrix uniformly, or pointwise, or what...? over the parameter space.
Can you provide more details on how you'd compute this integration numerically and how you would be able to obtain the conditional variance?
In the Kundu et al 2010 paper you linked, the expression for this expectation is given in page 118. What is the nature of your difficulty with it?
Do you understand the likelihood of the $\\mathrm{U}[0,\\theta]$ model? Can you graph the likelihood? What is the sufficient statistic? Considering the two hypotheses, can you write the likelihood ratio?
Thank you very much!
Just one question: how does kcca methods deals with number of starts (is it optimises the analysis with regards to the starting points)?
Have you tried fitting a simple model like $v_k = \\alpha v_{k-1} + \\epsilon$ to your data?  I'm all for neural networks, but I'd try a simple model like that if just to give yourself a baseline for comparison before looking at NNs.  Simple $AR(k)$ models are good for things with oscillations too.  Once you've hit the limits of this sort of "white box" model, then I'd think about NNs.
Do you also know the sample covariance matrix of $\\hat{\\beta}$?
and will play few hands, (or will correctly play few hands due to a dearth of playable situations) so for a long time the mode will be the initial stack minus the total of the antes. By the way, many poker sites let you see the stack distributions of tournaments in progress, so it is relatively easy to get data. However, I don't think equities are sensitive to the distribution until you have only a few more players than prizes.
If the differencing factor is over 1, an intercept induces a quadratic trend or worse. Who would want that? Also, the link you provide does not discuss auto.arima.
how about `c(rnorm(75, 1, 5), rnorm(25, 2, 4)`
Assume this is leave-one-out. Each predictor in each fold already has a confidence interval, e.g., from a Bayesian posterior CI or std err from `glm(..., family="binomial")` in R. What do I do with the intervals for each predictor across the leave-one-out runs?
By additive bias I mean predicted = observed $+$ constant and by multiplicative bias I mean predicted = observed $\\times$ constant.
Example 6 of [this paper](http://www.stats.ox.ac.uk/~doucet/doucet_godsill_andrieu_sequentialmontecarloforbayesfiltering.pdf) seems to match the form described in the addendum.  Any other comments on this paper?
Now I understand the assumption of sphericity and yes, that would be a mistake to do. But what is my missing data? And the repeated measure is not quite right, as but group A **is** before and group B **is** after treatment. So A and B differs in both time AND condition, which is the reason I doubted that the data is paired in the first place.
(+1) I like this because it relies on only the most basic properties of expectation, namely monotonicity. In case one is worried about what to do with the right-hand side, they can note that $\\max(1,|X|^r) \\leq 1 + |X|^r$. If one prefers an application of Jensen, they can write $|X|^r = (|X|^s)^{r/s}$ and note that $r/s \\geq 1$.
Good example. Nick is using R. (Until everyone uses R, it's good practice to mention that.)
@Micheal Lew - +1 i wasn't aware that Newman/Pearson had coined the phrase hypothesis testing, and I interpreted it in a rather more informal fashion. Also, can you please elaborate on how my answer is wrong, as I would like to correct any errors and am always eager to hear feedback.
Dear @louzer: Welcome to the site. I see this is your second question posted here. For future reference, please do not cross-post simultaneously to multiple SE sites. An identical version of this question exists on *four* different SE sites ([math](http://math.stackexchange.com/questions/621416), stats, [quant-finance](http://quant.stackexchange.com/questions/9813) and [MO](http://mathoverflow.net/questions/153070)). This is [not](http://meta.stackexchange.com/questions/64068) appropriate use of the StackExchange network or of your colleagues here. Cheers.
I'm curious to know the answer as well, and would encourage anyone answering to weigh in on whether it's important to address nonessential multicollinearity by centering before multiplying variables (including squaring them) in this case. I've been taught that it is, but I've read recently that centering sometimes makes no difference and never eliminates essential multicollinearity (that much I had been taught previously as well).
great link to the recommended question, as it has a thorough explanation of the background
I do not fully remember the details of your simulation, but if you do not simulate data with correlation, it should not make sense to add or test it as a parameter.
I have edited the question. Hope the question is understandable now.
Let's assume 2), i.e. the measurement error follows gaussian distribution.
Dear Yuval, you might consider registering your account as this will make editing (and other activities) somewhat less burdensome. Cheers.
Since Hopfield networks are autoassociative it's more useful to think of them as a value-value lookup rather than a key-value lookup, which would be heteroassociative. They can be used for denoising, filling in missing variables, or even approximately solving hard optimization problems (like TSP). I also wouldn't say SOMs are for classification (unless you would say k-means is for classification, which I definitely would not).
@whuber, So you mean that $C: x > 35$, since in this case we reject $H_0$.
Similar question: [What is the difference between Kalman filter and moving average?](http://stats.stackexchange.com/questions/16458/what-is-the-difference-between-kalman-filter-and-moving-average).
Ah thanks. If the sum is $c$, why does the sum reduce to $p(b|c)$?
I think this is probably the right answer as Mans T has formulated this about the variance of a population from which the vector X is a sample.   I find like both Mans T and procrastinator that a population and a sampling mechanism are not specified. The question is posed by saying "Is it meaningful to say that this is a significant result?"  That is vague because Var(X) is not related to an inference problem and nothing is said about whether or not the components of X are independent.
Is essentially what is being said in the  http://robjhyndman.com/hyndsight/crossvalidation/ excerpt that you should take a portion of the data and fit the model and the remaining portion of the data subtract the predicted value on the new observation from the true value  to get the error?
@whuber Thanks for the correction, I am not familiar with R syntax so I didn't know that %*% means $A^TA$. I removed my comment.
Please tell us some more. What is your N? How big is deviance etc? (Not just p value).
Noise in optical spectra usually depends on the intensity of the measured light ("counting" Photons is a Poisson process) and often also on the wavelength/frequency (because of detector characteristics). There is a whole bunch of processes contributing instrumental noise, see e.g. Skoog & Leary: Principles of Instrumental Analysis. The predominant type of noise will depend on the type of instrument (and the experiment). The d over time plot shows a clear dependence on the magnitude, suggesting that Baptiste has intensity measurements (as opposed to, e.g. absorbance spectra).
Can you expand on the statement that you were not able to incorporate the recommendation of including `(1|subject_id)` ?
OK I take notice!
@ttnphns: The only technical part of my answer (a numbered list with 3 items) is providing some proofs and can safely be skipped. The rest, I believe, is not particularly sophisticated! Maybe I should move that "extra" part down, as an appendix? Regarding your comments: I think this is a valid approach, and I like the ASCII looks of the SPSS "territorial map". Maybe you could move your comments into a separate answer (and give an exemplary picture of the SPSS map there), I think it would be helpful for future references. The results should of course be equivalent.
Unfortunately, basing cutpoints on the SD does an awful job when data are highly skewed: it will cram most of the data into one or two bins and leave a lot of empty bins.  When you use an arbitrary nonlinear method such as breaking into $[0,0.4]$ sigmas, then $[0.4,0.5]$ sigmas, etc., the result can be deceptive.  (When I see data presented in such ways it raises flags suggesting someone is trying to distort the summary or they are just ignorant of effective methods to summarize values.  You probably don't fall into either category so you don't want your work to be taken that way, either.)
Of course now the SPSS way can also be the R way: "Starting with Version 16, SPSS offers a free plug-in that lets users run R code within SPSS having full access to the active SPSS Statistics data, and writing its output to the SPSS Statistics Viewer" (http://insideout.spss.com/2009/01/13/spss-statistics-and-r/)
Thanks! +1 The graph is particularly helpful!