tttjjj/elt_bench huggingface dataset:
There is a body of theory that deals with the case where parameters of the distribution are estimated from the data before the KS test is applied. The reference is J. Durbin,  Distribution theory for tests based on the sample distribution function, SIAM 1973. Alternatively, you could use the Anderson Darling test for normality which can deal with the case where $\\mu$ is known and $\\sigma$ is unknown. There is a Matlab implementation of the latter called AnDartest, available in File Exchange.
+1 very clear explanation of the issues
@peterFlom The *actual* percentage change would be $(x_t-x_{t-1})/x_{t-1}$ (well, times 100% I guess); this is not the same as $log(x_t/x_{t-1})$ when the percentage change is large.
Furthermore, frequentist methods include restricted maximum likelihood. As Laird and Ware say, "the Bayesian approach leads to the unified treatment of estimation and computation" (vague priors for fixed, i.e. population, effects. "Random-Effects Models for Longitudinal Data", _Biometrics_, 38, 963-974) They refer to Harville, "Bayesian Inference for Variance Components Using Only Error Contrasts", _Biometrika_, 61, 383-385. See also Searle, Casella and McCullohg, _Variance Components_, John Wiley & Sons, 1992, §8.3g "A Bayesian justification for REML".
hadn't heard of him before, but I'll check him out. Thanks for the intro!
I think this might be a little misleading. (mean 1)=(mean 2) is a point null hypothesis, which in the Bayesian framework always has probability zero (??) (I'll have to go read the link you posted, but I have my doubts.)  I would instead direct the OP to @whuber's comment above ...
@guy That's a valid remark. Degrees-of-freedom correction has always been, for me, a conceptual issue with Method of Moments. After all the "sample analogue" is not a strict concept, and it is linked with the concept of "sample means" through the asymptotic correspondence of the latter with the expected value -but in an asymptotic framework, dividing by $n-K$ instead of $n$ does not make any difference. For me it remains an unresolved matter. On the other hand, the maximum likelihood estimator is concretely determined by the likelihood equations, and it may or may not coincide with MoM.(CONTD)
Can you just use a finite-difference approximation to get a good-enough estimate of the hessian ... ?
I'm exactly asking for which tasks/examples can the classification of such document images be used for !
I replied to you and @mbq in an answer to my own question.
Predictors that cannot help split the sample statistically significantly, to not appeer on the tree. But tree building has many options and regimes. Under other conditions those predictors can appear, and other can get dropped.
While that would very often find serious problems, the actual assumption is that the distribution is normal with constant variance within each *combinations* of IVs ... which is why people tend to examine residuals to check the assumptions. In fact, it's possible for the assumptions to all be fine (i.e. reasonably close to true, though not actually true), but just checking conditional on each IV one at a time to look quite poor for at least some IVs.
I disagree and agree at the same time. I agree that you need to carefully preprocess your data, to make the notion of "variance" have the same meaning in all dimensions. But I disagree that thinking with distances helps here. **With distances, most people do not do appropriate preprocessing either**; and it definitely doesn't "force" you to do so.
I am only trying to model and then predict stock volume, preferably by a day.  I'll look into time series analysis again.  Thanks.
A Bayesian estimator is defined through a loss function: it is the value which minimizes the expected posterior loss. When the loss function is the squared distance the estimator is the posterior mean. The mode appears as the Bayesian estimator associated to the 0-1 loss function.
With large enough samples, even trivial differences may be statistically significant. Is a difference in the third significant figure of any *practical* importance (how much difference could it make to you, really)? If there's no practical difference, there's no point testing for statistical significance.
Because the value of a beta can be anything (including of absolute value greater than $1$), it cannot generally be interpreted as a partial correlation.
+1 nicely explained. Concerning your 50% estimate: the chance varies with $i-j \\mod n$ and with $k$. For $i = j+1 \\mod n$ and $k\\gt 1$, that chance will be $1-k/n$ rather than $50$%. For larger distances between $p_i$ and $p_j$ the chance can depend (strongly) on the algorithm.
Don't you think that the Wikipedia article about it is enough?
They represent measurements time of a quantity on the same sample by three different devices. In theory they do not influence each other ...
Hrm, one really simple explanation:  When estimating multiple values, the extreme individual estimates are likely due to dumb luck and need shrunken to better match their true values.  A reasonable target to shrink towards is the average of all the individual estimates.
Cheers! This brings up a lot of great points. Interpretability was something I was concerned about and what you've written here regarding that makes a lot of sense. Thanks!
On a not very related point, you should know that ~10% of men are red-green colorblind.  Many people will not be able to read your figure *at all*.  If you were to send your paper out for review w/ 3 white male reviewers, there is a 1/4 chance that >=1 couldn't read it.  I would strongly advise you to use different colors.  Some additional info can be found here: http://stats.stackexchange.com/questions/16631/what-are-essential-rules-for-designing-and-producing-plots/21961#21961  Best of luck with your project.
Well ok, KNN is the first classification algorithm that I used. It is very simple, there no really a learning phase, the model is just a copy of the dataset.
thanks , the source gave me a good explanation for a start
Can you provide more details? I'm not sure to see how this answers the question given the information available in the question (that may be just me, because I'm not familiar with such topics).
[These quotes](http://en.wikiquote.org/wiki/George_Box) are very appropriate. All models are wrong, but some are useful.
His sample size would be too small with < 10 datapoints versus the amount of parameters you need to fit in Granger.
How do you determine the counts? I ask because you may need to normalise the count data depending on the measurement system
To answer the question one has to establish a quantitative understanding of "better." Some common meanings are (1) in terms of average error in approximating individual Binomial probabilities, (2) errors in approximating tail probabilities, (3) errors in approximating central probabilities. All these errors can be assessed in different ways--as absolute differences, relative differences, with various weights, and so on. Then the scope of $n$ and $p$ must be specified: what will their typical values be? In most cases I believe the truncated Normal will actually be *worse.*
The two sample KS assumes independence. That doesn't seem to be the case here.
Can you clarify what you mean by $x^t$? Are you *raising* some variable to the power of the trial number? Or are you *indexing* some variable by trial number (i.e., $x_t$)?
Longer intervals have longer weights. (The longer each individual interval is, the more varied time is.)
Can you clarify this?  I don't understand the setup or the question.
Wow! Thanks Glen!

Is there a easy way to estimate the parameters using for example R or Matlab or any other statistical software?
@RiaGeorge Q is obtained by taking the logarithm of Eq. (6). To find the maximum, one has to find the values of $\\sigma_{c}$, $\\sigma_{w}$, and $\\textbf{h} = \\{h_{1}, \\ldots, h_{N}\\}$ for which all partial derivatives vanish. A sufficient condition to ensure that this point is really a maximum, rather than a saddle point or a minimum, is that the Hessian matrix is negative definite.
Hi Zoe, I cannot understand how your data exactly look like, can you give us some example? You don't need to freak out anyway, your data is very simple and there is definitely a test to give you the results you need. Don't forget to include an example of the desired outcome too.
@DavidMarx: Sorry about that.  I've flagged this question as a duplicate.
No need to apologize.  This is a good question, and it will *not be removed*.  It will simply be *migrated* to another website where you will likely receive better answers than what you would receive here.
Can you define sighting effort and how this influences the perceived number or likelihood of spotting a species? Are you taking advantage of the geological nature of these data (i.e. using neighboring values to infer the distribution of values in close by cells)?
This is more than possible; if you construct a hierarchical model, this is common practice. Are you able to share more information about the problem you are trying to solve?
@Ken, whoever told you that is badly out of date (and wrong, which is worse). See, for example Zimmerman 2004 "A note on preliminary tests of equality of variances" or Rasch 2009 "The two-sample t test: pre-testing its assumptions does not pay off". If you don't have a reason to suppose the variances are equal, use the unequal variance version of the t-test. 

SamuelJ, you could compute the LR statistic and use it in a permutation test, that would work, if perhaps inelegantly.
Bernd, thanks for your response.  It looks like the mvmeta package is very new (published on CRAN on August 24!).  The paper link references a Stata version of mvmeta, which seems to incorporate methods of dealing with mixed treatment comparisons.  I'll have to see whether that functionality was also ported over to the R version of the package.
@Iterator: note that it's all R standard plot functions, no ggplot2 involved :)
I doubt there is a formal derivation, hence the quotes around poor man's version of adaptive parameters.
Sorry I don't have a lot of stats knowledge but from my general understanding, the populations are dependent since the male population is a subset of the total population of subjects. If the populations were independent could I use a binomial approx and check for the result being within 3 std deviations? Sorry I am totally out of my element here, but I am trying to determine statistical significance for a bunch of different usability tests where I want to know if the subset of users proportion is statistically significant to the larger population of subjects.
I am having trouble making sense of the first two inequalities and the notation. According to the question, $X$ is a family of *random variables* indexed by $\\theta$: it is a *stochastic process*. Thus the integral is not a Riemann or Lebesgue integral at all and does not appear to be an expectation: it would have to be a random variable.  The meaning of $f_\\theta(\\theta)$ is mysterious. Could you please explain your interpretation, explain your notation, and justify your steps?
Happy if this helps... Yes, at $t_0$ the time spent in state $z=0$ contains information about how quickly we can leave it. One can draw a graph with the four states of $s_t$ as nodes and transitions as
arcs. Then $z_t=0$ means that $s_t$ is in the group of states $\\{00,\\,01, 10\\}$. Within this group, $s_t$ can be one step away from
the exit $z =0 \\rightarrow z=1$, or two steps away.
Maybe I'm not seeing something, but why does "averages of the cumulative sums" make sense to you? On the horizontal axis you have the average of each trial #, and the vertical axis you have the cumulative population. What is this supposed to communicate? To me, it would be meaningful if the *difference* in samples would be shown. It is important to know if the population in trial#1 is much larger/smaller than the population in trial#37 Cumulative doesn't make much sense to me because it doesn't show me anything.
You can always orthogonalize the generated splines (for example the `rcsgen` Stata command uses Gram-Schmidt orthogonalizaton)
This answer is misleading. When the number and sizes of the clusters are known a priori the problem is known as *graph partitioning*, not graph clustering. The wiki page is not great, but a start: http://en.wikipedia.org/wiki/Graph_partition.
I believe the crux of the matter here is not applicability of tests but rather their meaningfulness and interpretability.  The question refers to "respondents." This strongly suggests the possibility of a nonzero non-response rate.  Even a tiny non-response rate (a small fraction of one percent) relative to the study size would amount to an enormous non-response rate relative to the small subgroup. That calls into question the representativeness of any subgroup this small.  As a result, it's a huge hindrance to any statistical analysis.
thank you for your response. I talked to a professor on my school and he suggested me to look for a way of calculating the confidence interval of your co-variance.. I ran into this http://www.talkstats.com/showthread.php/12192-confidence-interval-for-covariance-estimation.. If I understand it correctly, co-variance= \\hat{s}, the confidence interval for \\hat{s} is,  [\\frac {\\hat{s}} {\\chi^2_n(1-\\frac {\\alpha} {2})}, 
\\frac {\\hat{s}} {\\chi^2_n(\\frac {\\alpha} {2})}]
`lm(liking~picture*info)` or `lm(liking~picture+info)` for models with & without interactions respectively.
Do you have to use an SVM? It's possible that this could be easier with another approach.
Thanks. It seems to be a difficult thing to do in R. Maybe I'm wrong.
Do you mean they suggested a transformation $Y^* = Y/\\sigma(x)$ in order to make the variance of $Y^*$ constant?
The dependent variable, however, stays the same? How many independent variables do you have in each model? I think you can use AIC (Akaike Information Criterion) for comparing your models, as long as $Y$ stays the same.
"the frequentist IS BOUND to lose " Looking at the most-voted answer, I'd assume this depends on the utility function (e.g. not if regret optimization is going on). Intuitively, it might also depend on the ability to determine the prior function...
An interesting aspect of this plot--a similar issue was overlooked in at least one early textbook on circular statistics--is that the radial portrayal visually exaggerates the probabilities in the outer extremes, in direct proportion to their distance from the center of the plot.  When this is coupled with the (huge) loss of perceptual accuracy in reading values in any pseudo-3D graphic, one is left admiring the pretty computer construct but wondering about it actual utility (and shaking one's head about its implicitly large Lie Factor).
A good summary. Pages 48-53 ("long rambling cynical diatribe") of the following paper provide more details on that (the paper includes some results for higher order n-grams as well) http://research.microsoft.com/~joshuago/longcombine.pdf
Since I don't have your data, I can't reliably diagnose what's going on here. What I think it is is your algorithm is computing fitted values using a reduced / singular matrix. Have you diagnosed your data for multicollinearity? I would try adding your predictors one at a time to see where this algorithm breaks, and also running it with an example to see if there aren't backward compatibility issues.
I disagree for the reasons outlined in my answer on October 5th and Kochede's answer on December 23d.  To study variable relationships, you have to look at complete economic cycle and as much data as you can get, not cherry pick only recessionary periods (they are all quite different).
@user2079802: No, for $x_2$ through $x_{39}$ you get two terms in the exponent: $\\pi(x_i\\mid x_1,\\dotsc,x_{i-1}; x_{i+1},\\dotsc,x_d)$ $\\propto$ $\\exp(x_{i-1} x_i+x_i x_{i+1})$.  But it's still easy enough to evaluate that for $x_i=\\pm1$.
If you read this old book, you should also be looking into Åke Björck's *[Numerical Methods for Least Squares Problems](http://books.google.com/books/?id=ZecsDBMz5-IC)*, which has stuff not discussed in Lawson/Hanson. The routines included in the Lawson/Hanson book are [available from Netlib](http://netlib.org/lawson-hanson/).
(cont'd) I don't really understand your comment about p-values and multiple testing. Is there a contradiction between using p-values as evidence and adjusting for multiple testing? And your final comment about the evidence in a p-value from a t-test depending on sample size: isn't the degrees of freedom adjusting for sample sizes? And besides, how could this impact this particular data set when the sample sizes are the same for all comparisons?
@Marius: to clarify, are you suggesting `boxcox()` on the raw data, or on the transformed data?
In your example, $n$ would be 100 and $\\hat p$ is 0.3 Then you could use the textbook formula to get a confidence interval around the 0.3
@Ben another consideration in choosing which level (small firms vs. small + large gov't) of your variable to use as a baseline is the number of observations in each group. If you choose a baseline with a small number of observations the SE for the model intercept will be large, making comparisons to the baseline less precise. Comparisons from one group to another shouldn't be impacted.
With just "two time points" you can describe the entire dataset with a 3 by 3 table (giving the counts of the nine possible transitions), which is hardly a challenge to display or visualize!  How does your situation differ from this?
What are those conditions? And how does his theorem differ from the paper I cite?
Thank you very much! Your answer was very helpful. I understand now what I was doing wrong. Best regards, Maria
@Christopher I'd recommend that you update your question to reflect your above comment. Also, for correlation to be meaningful, more than just equal dimensions are required; the actual measurements have to come from the same cases, which in your case is presumably the same time points.
This question appears to be off-topic because it is not about programming.
Comparing votes at a sample of polling places to votes at other polling places is not a reliable way of detecting fraud.  If a modest amount of ballot stuffing has occurred in all of the polling places, or in a significant fraction of them, then you're not going to see any statistical anomalies; they're all going to look identical.
"What is wrong with the approximations that already exist?" I don't know what they are? Which is why I'm asking about them?
It entirely depends on who your audience is.  Many would be wowed by the fact that you'd done a regression at all; with only a vague understanding of what that is.  Some (perhaps like me) would raise an eyebrow but admit to themselves it probably doesn't matter.
Could you show a sample of your dataset?
**Normalising** typically means to transform your observations ${\\bf x}$ into $f({\\bf x})$ (where $f$ is a measurable, typically continuous, function) such that they look [normally distributed](http://en.wikipedia.org/wiki/Normal_distribution). Some examples of transformations for normalising data are [power transformations](http://en.wikipedia.org/wiki/Power_transform). **Scaling** simply means $f({\\bf x})=c{\\bf x}$, $c\\in {\\mathbb R}$, this is, multiplying your observations by a constant $c$ which changes the scale (for example from nanometers to kilometers).
The problem is not with you: I'm afraid those slides don't say anything at all.  They are too vague to give anyone a definite idea about what they are describing.  Try to contact their author for clarification.  Better yet, tell us about the actual task you need to perform and where you're stuck.
Have you ever read *How to Lie with Statistics*? It was published decades ago but it's still an enjoyable read...
+1 For adding a useful and thoughtful perspective to the question.
My sense is that this, coupled with Angrist and Pischke, would provide the most up to date perspective, although both are geared more toward micro analysis, rather than the types of time series or time-series cross-section analysis that macro analyses entail.
@ACD read the first sentence [here](http://en.wikipedia.org/wiki/P-value) -- 'at least as extreme', rather than equal.
@Ralph, +1 for looking up the reference. I should note however that if this is truly how the authors are dividing stochastic part it is a bit misleading. Suppose you have autoregressive model $Y_t=\\rho Y_{t-1}+\\varepsilon_t$. The division is that $\\rho Y_{t-1}$ is systematic and $\\varepsilon_t$ is non-systematic part. But if we substitute $Y_{t-1}=\\rho Y_{t-2}+\\varepsilon_{t-1}$ into the first equation and repeat the process indefinitely  we will get that $Y_t=\\sum_{s=0}^{\\infty} \\rho^{s}\\varepsilon_{t-s}$, so there is actually no systematic part according to the original definition.
if you are using `with()`, then go the whole hog and use `within()` here --- gets rid of more `$`. Your first line could be, `tst <- within(tst, tmn <- ave(Objects, SessionNo.))` whilst your second line could be `tst <- within(tst, devmn <- Objects - ave(Objects, SessionNo.))`. Note that the second line contains two modifications as the whole thing can be inside `with()` or `within()`.
That's way heavier-tailed than exponential. If it were exponential, a power transformation between cube root (i.e. Wilson-Hilferty transformation of a gamma) and fourth root (an alternative that can work better for a gamma with a small shape parameter) would nearly symmetrize it and render it pretty close to normal. I doubt fourth roots have much hope of working on that data though. But I think it's better to focus clearly on what questions you need to answer from your data and ask about *those* than come up with your own guess at a solution and ask how to implement it.
I believe there is no general term for "some combination" of distributions simply because the concept is far too broad to be useful.
This question includes additional material on a closely related subject: http://stats.stackexchange.com/questions/24242/  In fact, doesn't it answer your prediction question outright?
For good ideas, study some comic books ;-).  Seriously.
@Benjamin Thanks - that paper has a ref to McCabe 1982/1984 http://www.stat.purdue.edu/research/technical_reports/pdfs/1982/tr82-03.pdf which uses the term 'principal variables' for exactly what I'm trying to do. That term didn't seem to catch on though...
That depends on the research question. Are you interested in seeing how often you hit the bullseye or how far you get from the bullseye on average? In either case, if you just measured the distance it would leave your options open.
In general, linear regression techniques don't have much to say about $x$, just about $y|x$. There's nothing in either model that says orange has to be on the left and blue has to be on the right.
I just add the picture to show the result of t-test. Please kindly help me interpret the result. Millions thx.
I don't think clustering is going to be useful here. You might look into network analysis.
Is it later yet? =)
Maybe this can help: http://www.research.ibm.com/PM/sigt98.ps and http://www-bcf.usc.edu/~liu32/cs599/Time_series.pdf
But in the context of regression you should not be abbreviating with "IV", which is usually reserved for "instrumental variables", not "independent variables"; at least among economists.
@Elvis I guess not, but that is what it is called in these lecture notes: [see pages 24-25](http://ocw.jhsph.edu/courses/MethodsInBiostatisticsI/PDFs/lecture9.pdf)
Do you have any other siblings?
Additionally, suppose someone on here proposes a test and the analysis does come out at p < 0.05.  What you've done is just take two passes at the same data looking for the same thing with an alpha of 0.05 so now you've actually inflated it and have to correct for the multiple tests.  Stop thinking of your statistics and research that way.  It's the path to Type I erros.
All required calculations are provided in [my paper](http://hal.archives-ouvertes.fr/docs/00/64/73/61/PDF/IntrinsicPoissonRates_v3.pdf) but I have not studied the case of $H_0:\\{\\phi=1\\}$ ($\\phi$ is the ratio of the two Poisson rates)