tttjjj/elt_bench huggingface dataset:
Good answer, Karl, but could you explain *why* it's valid to adjust only for the four interaction tests?  There also appears to be a complication because the four ANOVAs clearly are dependent: they share the control data.  It's unclear then how to adjust the p-values.  Shouldn't this testing be done with a single ANOVA, protected by an overall F test, followed by a set of four contrasts to test the interactions?
Could it be that you actually want to fit the model $y = \\alpha  + \\beta sin(x) + \\epsilon$? If so, note that the model is still linear in parameters, i.e. there is only $\\alpha$ and $\\beta$, and for instance no weird function of $\\beta$. The non-linearity in $x$ is not really a problem. Also note that strictly speaking, the model is linear in $sin(x)$.
@ttnphns I'll stick by the "merely," thanks. Whether or not the implications are "profound," the fact remains that standardization of a variable literally is an affine re-expression of its values: a change in its units of measure. The importance of this observation lies in its implications for some claims appearing in this thread, of which the most prominent is "covariance-based PCA is the only truly correct one." Any conception of correctness that ultimately depends on an essentially *arbitrary* aspect of the data--how we write them down--cannot be right.
How did you find that the median group is group B? Should it not be the group D (14-20)?
@Aniko I agree with you. My comment was not a critic, and reference to optimal scaling was mainly for the case where the assumption of an *a priori* continuous scale of measurement is not tenable--because in practice, we often rely on Likert items without making strong assumption on the underlying construct; I also agree for Mann-Whitney test with ordinal variables.
"More random": how "randomness of _one_ time series" is defined here, and then quantified, so that one can measure the randomness of two or more time series in order to compare them?
@Ken Thanks. The `aplpack` package has also some good add-ons for data viz. However, the [rpanel](http://cran.r-project.org/web/packages/rpanel/index.html), which also relies on tcl/tk, is probably a better option for more complex stuff. Now, with RStudio and the [manipulate](http://www.rstudio.org/docs/advanced/manipulate) package, it's also easy to enhance basic plot in R.
Sorry, but what are loadings (c in your formula) and how do you determine them?
There are many ways that two distributions can differ. If you choose *one* measure, you may find no difference on that measure even if they still differ quite a bit. I'm in favor of using a single measure, *if it captures what you need it to*. The question is - will it? Something like the two-sample K-S picks up general differences. The partitions of the chi-square pick up general differences more weakly overall but let you choose a subset to focus on (notionally related to low order moments). If you have a specific measure you're confident will capture the differences you should use that.
Thanks a lot. Both models worked. Thanks again for your help.
Could you perhaps summarize the main points of that paper so that this reply remains self-contained.
Can you do all 5 stimuli in one session for each subject? Even better is to do Baseline-Stimulus1-Baseline-Stimulus2-etc. Then for each subject you should see the power spectrum change then return back to baseline. This would be very convincing even with a single subject.
(+1) I always check the number of downloads and the date of the last update ... if I do not have enough time to play, all small and abandoned projects are immediately dropped. Aside, reading the code of others is one step to become a better programmer.
Check out their page on data splitting. http://caret.r-forge.r-project.org/splitting.html
thanks @whuber. To add, given a dataset where most values follow a normal distribution but where there exists a potentially high proportion of outliers, it will provide an excellent robust measure of the variance of the normal distribution. If the time series data is can be assumed to have a constant mean and variance over time, then the MAD approach can be used to estimate distribution parameters from a sample of the data and in this special case there is no need to have a running median to estimate the mean locally. I guess any implementation will depend on the actual data characteristics.
Thanks for your answer! I'm not 100% convinced though: You say that "RSS(A) contains more than just the residual error, it also contains variability due to known factors." But this depends on what the correct model is. Perhaps $B$ and $A:B$ have no effect - we don't know this, it's just a hypothesis we're testing. And in addition to the hypothesized influences, there could be unknown ones. So how do we justify a-priori which model is closer to the truth? In regression, the situation is equivalent. Do you have some literatur sources which I could consult?
Welcome to the site, @dumdidum . Simply saying "tools that actually make sense" is not enough for an answer. If you could say a) Why the Dixon Coles tools don't make sense and b) What tools would make sense then this would be a good answer. Otherwise, it should be a comment.
Liked the suggestion about the note. Thanks for your help.
I am not a statistician and don't play one on TV either, but I would be highly suspicious of the results of a researcher who concluded from the above data that alcohol consumption is related to gender. Very close to $1/3$rd of each population falls into the three categories...
+1 to everyone commenting. Poster has to decide: if the distribution is lognormal, it can't be confined to a finite interval; if the support is a finite interval, the distribution can't be lognormal. One assumption must be wrong.
With just one dependent variable, your model is more simply described as _multiple_ regression rather than _multivariate_. Indeed, increasingly "multiple" although a harmless term is superfluous; having two or more predictors is not a big deal. Note that "IV" to many means instrumental variable; it is by no means a universal abbreviation across statistical science.
(1) Why do you think using "Log P1 - Log P2" is better than just "P1 - P2" ?

(2) Is what I've called P1 the same as P* in your explanation ?

(3) What do you mean by log P(x) ? is it same as Log P
Are you sure that you are doing a univariate-time series modeling? I am asking this because at each time point, you have 86 different measurements of your dependent variable (stockVol0). This reminds me a multivariate-time series modeling. I am not sure if SPSS can handle this type of modeling.
I think @cardinal has a point.  All but the last two paragraphs of this question are covered well at http://stats.stackexchange.com/questions/6652.  The question implied by the last two paragraphs exactly echoes http://stats.stackexchange.com/questions/3911.  Other questions that are *very* closely related include http://stats.stackexchange.com/questions/11609, http://stats.stackexchange.com/questions/2356, and http://stats.stackexchange.com/questions/2272.  If you can formulate a question on this topic that genuinely differs from these other five, please do so!
@Alexis I'm aware of TOST or statistical tests of non-inferiority in randomized clinical trials. I thought it would be clearer if the OP could clarify in plain English what he/she wants to do.
Hi, Thanks for your reply. I think I did not explain well. At a time, I am looking at 10 values and then I have p-values from dixon test from these 10 values. So I did this, for 50k rows but I applied test only on 10 values (row wise). So I know if p-value is significant (say <0.01) then I know there is a significant outlier in that row but I want to know which one (or more) is outlier. Do we have any test or procedure for this?
Do you have multiple target variables? Confused by the notation y_{i,1}
@Flask Probably an update of the `lme4` package.
The end result of the project that started with this question can be seen in the interactive web tool linked to from here: http://www.mbie.govt.nz/what-we-do/business-growth-agenda/regions/
Thank you very much -- your suggestion worked very well indeed. Also, thank you for your pointer to ezMixed, which is also a huge help -- I wasn't aware that ez offered support for mixed models.
(Actually, while I'm at it: Thank you for your work on ez, which is a great pleasure to use :-) )
I don't think you and the original poster are talking about the same thing. I think the OP is asking about the seasonal period, not the size of a coefficient.
You should supply more details. I see no way for anyone to reproduce your results nor any obvious way to guess whether the issue might be with the data or the algorithm or something else.
Can you give some more precise details of what you're doing, the sizes of the variables and so on?
OIC.... Well, then I'd go with the paired t-test or (equivalently) a one-sample t-test of the differences in scores.
Thanks to everyone.
Why don't you post your data and perhaps someone can help.
" I would like to filter out everything else and keep only the relevant information, or something along those lines,"

This is *extremely* ambiguous and will prevent anybody from trying to address your question. Second of all, why are you comparing numbers visually instead of arithmetically? The whole reason we have numbers in the first place is because visual inspections are subjective and prone to error.
@forecaster .... Continuing .... SCA has the capability of incorporating innovative outliers which (if present) AUTOBOX proxies with additive outliers.
(cont'd): Also note that they are using the beta in a *sensitivity* analysis. The sensitivity depends not only on the mean of the input, but the model's response to changes in that input. In general, the function of the mean of a random quantity does not equal to mean of a function of a random quanity: $f(E[X])\\neq E[f(X)]$, so while simply drawing proportions and averaging them will be insensitive to outliers, the *model's* response to such outliers may not be linear and symmetric. Also, its good form to not allow your model to take on unrealistic values, inviting more criticism
BTW, looking at your graph you added to your question, note that you can view the area you want as the sum of areas of rectangles each of height $\\frac1n$ and widths $1-y_1, 1-y_2, \\ldots, 1-y_n$. Thus, the area is exactly equal to $1-\\bar y$, just as I answered originally.
@Manoel as the reply by @chl shows, this question is fundamentally statistical in nature.  If questions like this were off-topic, then so would be about half of the questions on this site!
Because the square root of a $\\chi^2(1)$ variable is half normal, most applications involving chi squared values with 1 degree of freedom can be interpreted as applications of a half-normal model.
The task can begin with exploratory analysis. Data collection serves more than hypothesis testing, and [the scientific method](https://en.wikipedia.org/wiki/Scientific_method#Elements_of_the_scientific_method) does not necessarily begin with a hypothesis...not that you necessarily intended to imply the opposite of these points.
can you be more precise? "percentages" are "numbers", after all. I, for example, find difficulty to interpret your question.
The R tag seems unnecessary here. The question transcends particular software choices.
@ocram You were right, I have tried and results with test data indicates "overfitting".
You're forming a confidence interval for the *parameter* $\\mu$. That confidence interval applies equally whether you're looking at the variance, the mean or any other population quantity which is equal to $\\mu$. Your last paragraph suggests you're confusing the parameter and a sample statistic. A confidence interval for $\\mu$ in the normal (which is the population median) is just a confidence interval for $\\mu$, however you construct it. When you say 'median confidence interval' do you mean 'a confidence interval for that median, $\\mu$ based on *sample quantiles*'?
Well, that's why I have put the example of accidents count. So, you see you can not ignore the chance of occurring two accidents in our study area at exactly the same time (very rare though). But for the ease of further calculation the assumption the OP pointed is made.
Thanks for the answer! I would upvote it if I had enough reputation.
The ridge/LASSO make the assumption that $E[Y\\mid X] = \\beta^T X$. Gaussian processes let $E[Y \\mid X]$ be (essentially) any function. If I model $E[Y \\mid X]$ using Gaussian process regression, I do *not* get an estimate of the distribution of $Y \\mid X$ from only the GP, I get an estimate of the posterior distribution of $E[Y \\mid X]$ which is a **random** function in the Bayesian context (i.e. it is not fixed, and so has a posterior distribution).
Thanks for clarifying things!
I think we need more detail to be able to help. How big is the initial cross-tabulation (e.g. 2x4 table)? How are you doing your post-hoc tests? And in which statistics package? I don't think you should need to look at standard residuals in this context either.
+1 but as a minor nitpick, you do need to assume that $E[X^3] = 0$ separately (it does not follow from the assumption of symmetry of the distribution or from $E[X] = 0$), so that we don't have issues such as $E[X^3]$ working out to be of the form $\\infty - \\infty$.  And I am queasy about @ocram's assertion that "**a** N(0,1) rv and **a** chi2(1) rv are uncorrelated." (emphasis added)  Yes, $X \\sim N(0,1)$ and $X^2 \\sim \\chi^2(1)$ are uncorrelated, but not **any** $N(0,1)$ and $\\chi^2(1)$ random variables.
Ah, I see that I misunderstood you. And you have resolved the problem (with this example) for me: the (short term) solution being that I need to evaluate convergence for abs(SUMstep). The longer term solution is, as you rightly point out for more attentive programming.
The pragmatic answer remains: if you have the interaction in one model, then people will ask you why it isn't in the other, and the best way to deal with that is to just include it and show that it is not significant.
Can you be explicit about what 'combined' means? Not just in words if possible.
(+1) This is a good find. I've added a stable link to the Daniels paper as well as another reference that seems to complement it.
Do you know what the mean and variance of sums and differences of variables are, given their means, variances and covariances? i.e. what $Var(X+Y)$ is, for example? Do you know a fact about the distribution of linear combinations of jointly normal random variables?
Does anyone know how to add an attachment? I did not see that option. I saw for links but not attachments. The formatting has changed for the table. It's jumbled up now :\\
Your Tikhonov-regularized least squares is perhaps more commonly called [Levenberg-Marquardt](http://en.wikipedia.org/wiki/Levenberg%E2%80%93Marquardt_algorithm) in statistical circles, even when applied to pure-linear problems (as here). There's a paper about online Levenberg Marquardt [here](http://www.cee.uma.pt/morgado/down/lm_online_final.pdf). I don't know if that's any help.
Thanks, Peter. This approach does not fit with my analyses since I have female as the IV in other models. I appreciate your help.
Chi-square partitioning is extensible to multi-way contingency tables.  Here's the article that Agresti cites in his book, in fact...

H. O. Lancaster (1951) "Complex Contingency Tables Treated by the Partition of χ2" Journal of the Royal Statistical Society. Series B (Methodological), Vol. 13, No. 2
Thanks @carlosdc, but I am afraid that you're trying to teach this old dog, new tricks. A lot of that is way over my head. I read Mercer's conditions, but their meaning in the real world is lost on me. I assume from the above that the integral of a kernel must be constrained to a finite value. Is that assumption correct?
Thanks a lot. It's exactly what I was looking for! Perfect.
Next time I will provide a reproducible example.
Go argue with the SAS authors and with Hedeker and Gibbons, who I cited. I'm done.
Second the endorsement of Bayesian Data Analysis.
Thanks for your reply! I have some reading on probability densities to do..
@PeterFlom: Amazon doesn't have vol2's.
Machine learning and data mining are entirely within the scope of this site.
I agree with you and about a decade ago collected a bunch of such problems for a course (see http://www.quantdec.com/envstats/homework/class_03/paradox.htm ).  However, there is a strong pedagogical counter-argument: Probability itself can be confusing, so if you start off with counter-intuitive examples, you risk losing your audience forever (like Augustus DeMorgan, a pioneering probabilist, who later in life completely gave up on probability as hopelessly difficult!).  So caution is in order here, especially if you want to *motivate* people in an *introductory* setting.
Have you noticed that $X+Y\\sim N(\\mu_1+\\mu_2, 3)$ and $X-Y\\sim N(\\mu_1-\\mu_2, 1)$ are uncorrelated and jointly normal, whence they are independent? Thus you can digest your dataset into $\\{(X_i+Y_i)\\}$, view it as a set of iid realizations of a Normal distribution with known variance and unknown mean, and ask how to compare its mean to zero. This is an elementary textbook problem with a well-known answer (a Z test).
You can threshold continuous data. For example, the model underlying ordinal logistic regression (a.k.a. the proportional odds model) is equivalent to thresholding logistically distributed variables with mean given by the regression function.
Can you distinguish the items or just their colors?  Say, for the blue items, do you know how many are just in sample 1, how many are just in sample 2 and how many are in both?  Or do you just know the total number of blue balls and of the numbers in each of sample 1 and sample 2?
I don't see the point why do you need to control for a confounding variable with Dirichlet distribution.
Thank you so much. These replies are all very helpful.
I've added some code. My first suggestion was to include number of ratings, not as a predictor but as a response/outcome variable. I've added a second suggestion where the number of ratings is included as apredictor on the "group level".
There seems to be some confusion, at least about notation. If you know $\\mu_x$ and $\\mu_y$, you know whether $\\mu_x = \\mu_y$ without testing anything. On the other hand, if you do not know $\\mu_x$ and $\\mu_y$, you cannot compute your test statistic. The test statistic should be a function of sample/data, but no sample is described in your question.
Perhaps you could clarify this question by indicating how the data are used to predict a value (and what value is being predicted).  What do you mean by "changing subset of node"?  What is the "output" of a network?
Basically, the basic panel data or time series analysis (ARIMA etc) methods you read about in textbooks all assume normality. There are special methods like INAR for count data, i.e., for time series that take on values like 0, 1, 2, 3, ... However, if your values are usually "high enough" (>20), you should be able to use the basic approaches and not need to worry about the more complex ones.
One of the conditions to apply Slutsky's Theorem is that one of the sequences of random variables must be converging to a *constant*.  See the third bullet beneath "Statement" in the [Wikipedia article](http://en.wikipedia.org/wiki/Slutsky%27s_theorem).
Perhaps you have some suspicion of a specific kind of systematic data loss?
To me, two-sample t-test seems applicable in both cases (if you assume that the percentage preference is normally distributed).
I'm not an expert in this area, so take this w/ a grain of salt, but FWIW, I've always understood *Fuzzy Sets* to differ from *Probabilistic Sets*. Prob is still a kind of crisp logic, where an element is either in a set or not, but the best you can do is state the probability that it is in each set. Fuzzy means that the element is in *both* sets to differing degrees. [This section](http://en.wikipedia.org/wiki/Fuzzy_logic#Degrees_of_truth) of the Wikipedia entry seems to be saying it the way I remember it.
To add to Peter's excellent advice, failure to use subject matter expertise will result in such problems as finding that an "important" interaction terms is explained away by a missed main effect.
See the (oddly-named) Wikipedia article [Rule of three (medicine)](http://en.wikipedia.org/wiki/Rule_of_three_%28medicine%29)
Sorry, but I'm not entirely sure what you mean. However, if you're concerned about having multiple models, and just want one, be aware that changing which factor level is taken as the baseline _does not_ change the underlying linear model. i.e. you need to make two models, one to test A vs B and A vs C, and another to test B vs C, but they are the _same_ model. You'd only need to report 'using' one model, and present one set of results. e.g. take all results from the summary() of the first model, and just add relevant details of the B vs C test from summary() of the second model. Helpful?
I am not sure if he covers this, but a good book on simulation is Simulating data with SAS by Rick Wicklin (he deals with SAS but the methods should be exportable). Again, the book is at the office and I am home, but you can take a look at the TOC and see if it will help.
Given you have specified an exponential distribution, the most obvious thing to do would be to work with GLMs. However, there's no particular reason you shouldn't use nonparametric procedures; they should work just fine (though it may help to remember you're dealing here with scale shifts in an exponential, since the mean is a scale parameter; if you work with logs you can use the nonparametric approach to estimate an interval for the location-shift in the logs, and hence an interval for the scale shift when transformed back)
It has been pointed out that this is similar to a two-armed bandit problem where the rewards for each arm come from same set of payoffs.
or mark them as `NA`: `foo[foo == 999999999] <- NA`
@John They are means of multiple samples.
I think you have to ask the question in a more strict way such as "is the number of attendants to congress in rainy days significantly different from those in non-rainy days"? For me the most straightforward question is: is the number of people attending significantly different when it's not raining from when it is raining? But given the nature of your dataset it looks like the cquestion can be much more complicated.
Thanks. Very clear answer!
For the concept of power, please [search our site](http://stats.stackexchange.com/search?q=power+analysis+hypothesis).  "Less powerful" has no intrinsic meaning; your second question can only be answered if you tell us to what the unit root test is being compared.
A bit more detailed context would help: what do the observations represent ? What is the confidence of the score ? Where does the error come from ?
+1, this is fabulously informative.
If this is for homework or self-study, you should add the `self-study` tag to your post.
@gung Yes, I understand the point about count vs ordinal.  At the moment, I am trying to replicate the paper http://ideas.repec.org/a/ucp/jpolec/v99y1991i5p977-1009.html and they use an ordinal regression.  I have also estimated count models, but that doesn't help me with this particular task.

Also, no, it's not that I just want R to do this, I'm trying to understand where the behavior is deviating from my expectations (because I suspect the error is on my part, not R).
You can ask for the parameter estimates and look at the sign of the beta.
OP should note that log(C) doesn't have a mean of zero, so on the log-scale expectations will be "shifted" on average. (This is unavoidable - it can't be unbiased on both the original and log scale)