tttjjj/elt_bench huggingface dataset:
Another method for estimating Bayes factors which is purported to be reasonable in high dimensions (though I haven't tried it) is Nested Sampling: http://projecteuclid.org/DPubS?service=UI&version=1.0&verb=Display&handle=euclid.ba/1340370944, http://arxiv.org/abs/1005.0157.  The second paper exploits the power of Hamiltonian monte carlo, a particularly efficient MCMC method for high-dimensional inference.
@PeterFlom I was thinking that this method could also incorporate prior probability by creating a map of weights/ prior probabilities that adjusts the values in the map of the data.
Looks like a new bug. I'll look into it.
:) Yes, I know. What I meant was that the log or ln of $Y-9$ did not approximate anything close to gaussian. Anyhow, it seems, after further reading, that for effect estimation I can trust my raw coefficients.
Thank you for your answer, unfortunately it sounds to complicated for me. Can I perform a mack-Skillings test in SPSS?
Consider $\\textbf{all}$ random variables with support on the real line that have $0$ mean and unity variance. We have seen that among those random variables, the one with density $p_0(x)=A\\exp(a_2x^2+a_1x)$ has largest entropy, and this is clearly a Gaussian density (complete the square on the exponent if you want).
See Peter's answer for good comparisons between models.
In practice, I work on a model corresponding the first case you mention but the corresponding community uses the second case-terminology and I am very uncomfortable with that. I guess that following the community is the best thing to do.
(1) "A *sample* is a set of items or individuals selected from a larger aggregate or *population* about which we wish quantitative information."  Snedecor & Cochran, *Statistical Methods*, Eighth Edition (p. 5).  (2) "Independence" in the sense of independence of random variables is not defined in the sense you use it here--for that, I think I can safely refer you to any authoritative statistical source, even [Wikipedia](http://en.wikipedia.org/wiki/Independence_%28probability_theory%29).
Good idea, thanks! However `integrate` seems not to work since `density(some data)` is not a function. I used `integrate.xy` instead and there seem to be problems indeed!
Don't you want a Bayesian *credible interval* instead? That is directly computable from the posterior distribution of $a$.
@Glen_b You are welcome :) BTW: you are right about the sign of the measure and interchange of some data. When I use -pairsC(x,prob)+pairsD(x,prob) instead of pairsC(x,prob)-pairsD(x,prob), I get +0.684.
As a word of caution, be careful with dividing by `def`, however--if it's a **random variable** then dividing both sides by `def` could completely change the **functional form** of your regression. Just throwing that out there...
This sounds like a routine textbook type question. Is this for some subject?
I think that dudi.mix() would do the right thing in case you have continuous and categorical variables, but you might want to post your question to [the ADEList](http://listes.univ-lyon1.fr/wws/info/adelist) to get a more rapid response from the ADE developers themselves.
I am not an expert on them, but you might want to look at partial least squares http://www.utd.edu/~herve/Abdi-PLSR2007-pretty.pdf
Are your predictors measured subjectively?  If so, you'll want to read http://stats.stackexchange.com/questions/1576/what-are-the-differences-between-factor-analysis-and-principal-component-analysi
I'll second the advice of getting a good textbook.  Singer/Willett is good; I also like Weiss (2005) "Modeling Longitudinal Data"; chapter 8 "Modeling the Covariance Matrix" has this specific information.
I agree completely with @NickCox. `help predict` might be useful: "predict can be used to make in-sample or out-of-sample predictions".
@Mars - I wouldn't say the annoyance was with your phrasing, and certainly not with you - but with the people I've seen say something like the stuff in that last part, sure. I have edited most of it out now, my editorializing on the issue was not to the point.
Sorry! I knew I messed that up as soon as I hit submit. Time for more coffee!
@Alexis - The Michaelis-Menten model..... i never thought I'd have to look at it again after my days in the lab finished up! Thanks for this suggestion - reading now.
Both methods are "correct."  Which one you choose will depend on which method you find to be easier to interpret.
Of course, there is more data in a coin toss than just the result - A sensible bayesian will still probably bet even, because of the weight of past data, and because the coin and coin flip looks fair. Unless, perhaps, you can't see the coin, or the coin being flipped. In which case you don't even know if the data isn't just forged to start with, and you may as well toss your priors out the window...
There are many possible solutions, but choosing an appropriate one depends on understanding what $Y$ represents, on why these coefficients must lie on a simplex, and on the likely form of the errors.  Could you include such information in edits to the question?
Good point -- thank you.
Thank you so much for your post. I very much like your stratified MC idea. I was wondering if you knew about any resources on it so that I could read up on it? Anything with an example in action would be much appreciated.
Thank you for your answer. What do you mean by "answer is completely dependent on the nature of t"? Lets assume $t$ is a temperature attribute with values in the range 0 to 100 degrees. How does this information help to choose an appropriate measure?
What hypothesis do you wish to test?  Whether some correlation is zero?  Whether they are all zero?  Whether they equal some set of specified values?  Something else?  Please note that we cannot tell you what assumptions you should make: those are determined by your data and your analysis objectives.  Is there a particular reason you want to perform a permutation test?
Thank you this looks very interesting.
"the bigger your dataset, the more chance for meaningless overfitting" - Actually, that's backwards.  The bigger the set of possible models, the greater the chance of overfitting (all else being equal).  The larger the dataset, the *smaller* the chance of overfitting (all else being equal).
Related: (**1**) [A resource on concepts underlying statistics, not the techniques used in applied stats](http://stats.stackexchange.com/questions/7165), (**2**) [What book would you recommend for non-statistician scientists?](http://stats.stackexchange.com/questions/421), (**3**) [Teach yourself programming in 10 years](http://norvig.com/21-days.html) (*an only slightly tongue-in-cheek suggestion*), (**4**) [Intro to statistics for an MD?](http://stats.stackexchange.com/questions/223). There are many other related questions on this site.
See answer below. My response was too long for a comment.
Andy, thanks for the answer-you hit upon a number of issues that were in my head, but I couldn't quite articulate.  Addressing the ordering isn't practical in my situation, because of a number of other contstraints (mostly related to the end users).  However, the multiple scales is a great idea, and I'm going to start working on that today.  Also, I really like the dot plot in your third point-did you create that in Excel or R?  Thanks again for your insight.
@whuber You are right. So, OP, tread carefully here.
Thanks for your detailed answer, that looks like an interesting method and useful for other things I'm doing. The key question for me here is overlap of shapes, so I'm not sure if your suggestion to use points is going to help me answer that.
I think this would be a good candidate for a Bayesian approach
You might make headway by viewing each AP *and its signal strength* as providing an update of a prior multinomial probability distribution for presence in the rooms.  (In fact, if your training coverage is reasonably complete, you could go further to generate a spatial probability distribution for the exact location.)  Bayes' theorem provides the update mechanism.
It really depends on a scope of your project.
I once fitted my model in R, validated it, but then ended up implementing it in Java due to various performance issues.
Nice answer, Robin!
Any method of EFA (except the so-called alpha FA) treats the input variables as the population, not sample, of items on the research "field". Everything (loadings, communalities, restoring of correlations) change when you drop some variables. So, you should rerun and rerun EFA until you stop dropping variables and are satisfied with the factors you've got. Then you can proceed to CFA or cross-validation.
But will it improve estimate of variance for $L_{est}$? $L^*$ has to be preatty independant of $L$, doesn't it?
Sorry, I did'nt get your point. My definition of accuracy here is Hit or Miss, so if correct we get 1 otherwise we get 0. Do you suggest different definitions?. Is there a way to link accuracy measurement with the distribution of the data?
Hi Glen, this is the question I posted the other day. I think I am almost done. Let the solution given by optim be called S1. Let the solution given by EM be called S2

S1 achieved better likelihood, but I fed S1 into the EM code, it converged to S2 again, this suggests there is something wrong with my algorithm? i.e. the log-likelihood was not increasing at every step?
That looks promising! Thanks a lot for your help.
@Bogdan Lataianu: I've rewritten it, though I couldn't see what was unclear about the original.
@chl Isn't Pearson's $\\chi^2$ an approximation to Fisher's exact? So they should give similar results. Concerning the marginals, I can only say that the subtypes are fixed, but the classification algorithm (i.e., variable X) is not constrained and is free to classify any number of patients as either A or B. But this is similar to many situations where I have seen Fisher's exact test be applied.
Thanks for the valuable information. I did try the log(cost) with the linear regression, but it did not help much. I will add the  interaction effects and observe what happens. I will also experiment with transformations again. I will keep everyone posted with my developments and findings.
I took a look at the documentation for your function, Greg. It certainly gave me a chuckle. Unfortunately this wasn't exactly what I was looking for--it lacks the sort of application that I'd like.
Thank you for idea. It was really helpful. :)
Thanks for the edit, Matt!
What is the clustering method you're using?
The linear relationship only holds when `dW = 0`, so I think it's just how the intercept/slope evolve over time when performing static linreg at each data point.  For now I'm interested in getting this to work with `dW != 0` so I'm going to ignore question 3 for now.
Thanks. I'm looking into generating the PC's using the formula now. One of the datasets was inherited from the last RA, so I'm not sure if the zRead and zSpell scores were calculated in the exact same way. I'll do a bit of digging to see if the common mean and SD would apply.
@PeterFlom  That is a good answer and a good book recommendation.
See [here, last sentence of first paragraph](http://en.wikipedia.org/wiki/Regression_analysis#Regression_diagnostics), and [here](http://en.wikipedia.org/wiki/T-test#Slope_of_a_regression_line) and [here](http://en.wikipedia.org/wiki/P-value).
I asked about the covariance of two products (x1x2 and x1x3) -- that's the nonlinearity.
I've rephrased the question. I'm looking for changes in the etymologies, i.e. histories of particular words. You could say it's evolution of definitions, yes, parts of the definitions at least.
It's more of a programming/algo learning exercise for myself.
el chef has a condensed answer over here -> http://stats.stackexchange.com/questions/146/pca-scores-in-multiple-regression. HTH
There's a paper by Lutkepohl where he shows that the volatility adjustment in log transformed series makes the forecast worse because the true variance is unknown
thank you for guidance. now my supervisor asked me to learn performing outliers to tackle this issue :( another new thing Is it necessary to learn AMOS?
i mean can't i perform it with SPSS? my structure is "mediation"
This looks like a good approach. I think you can do better by means of a little algebra to avoid the ratio problem. The test whether $y=\\alpha_0+\\alpha_1x$ and $y=\\beta_0+\\beta_1x$ intersect at $y=0$ comes down to whether $\\alpha_0\\beta_1-\\alpha_1\\beta_0=0$. Since $(\\hat{\\alpha_0},\\hat{\\alpha_1})$ and $(\\hat{\\beta_0},\\hat{\\beta_1})$ have independent bivariate normal distributions, we should be able to estimate the sampling distribution of $\\hat{\\alpha_0}\\hat{\\beta_1}-\\hat{\\alpha_1}\\hat{\\beta_0}$ without resorting to simulation.
Do you recommend a specific software?
Introducing the possibility of negative amounts is an interesting observation, but not needed for resolving the issue in your question. See my answer.
I don't face trouble you tell about. Correspondence analysis (both simple and multiple) treats all variables as nominal, whatever level of measure you had explicitly defined for them.
By saying power of the periodicity I mean that the xaxis will show the cycles/hour i.e. 1/24, then the y will show db/(cycles per hour) where dB is a power. But yes, the series is non stationary, so the amplitude of the periodic component would change in time.
Indeed, that's what it (or a standardised version) is, but it's way pessimistic for errors. You use Stata: look at some samples using `rt(1)` as a function.
This sounds like CW material.
The inputs in a gaussian process are some training points and normally we set the X* points over the inputs. Thus we have more X* points than X points and they span the same range. Maybe this couls lead sometimes to the decribed X=AX* + B behavior
@Graviton. Working on it. See http://robjhyndman.com/hyndsight/fpp-amazon/
Thank you for clarifying Max's comment.  The rationale here for including more lopsided terms is a negative one -- that otherwise I would compute a very small probability.  Are there any positive ones?  As a novice I have difficulty in interpreting the sum that include lopsided terms.
@godzilla What do you mean by "good" and "bad"? What makes you think that the second model is "better"? By the way: It is generally [not recommended to do an automatic model selection](http://stats.stackexchange.com/questions/20836/algorithms-for-automatic-model-selection).
Nah, this is just because the word "informatics" has completely lost definitive meaning. "Bioinformatics" was coined just for "biology made on computer", there is nothing deep in this.
@Flav FWIW, the Eureqa, A.K.A. nutonian or formulize, application is pretty good for this kind of thing:  http://formulize.nutonian.com/ . For your problem, it will not deal with vector `Y` very well, so you will have to run it thrice.
I will check the material, thanks a lot!
thanks for your answer. I'm traveling now and have very poor internet access. I'll check the papers you linked later as I can only see the abstracts from here. Anyway, after thinking some more, I believe my suggested theoretical model sucks and is probably wrong as it is not flexible enough to show an arbitrary difference $d > 0$ between the groups and an independent rate of decline $\\alpha > 0$ at the same time. I wonder if there any systematic method to build such theoretical models?
To Gabriele Pelli, Were you able to follow the answer? I am doing same test for two variables (FDI and GDP), but could not understand!
@whuber That's a fair point.  For a single categorical predictor looking at the variance and mean of the sub-groups would be sufficient to detect overdispersion, but for a multivariate Poisson regression, it is not. For the sake of argument, let's assume both a Poisson and negative binomal regression have been done and the negative binomial shows a better fit via anova model comparison. That should indicate overdispersion. Given that, how could the variance/overdispersion be modeled parametrically rather than as a constant?
Whenever it comes up, you are essentially comparing two means, like a t-test.  It would come up in the analysis of a general linear model.  My favorite book on multivariate analysis is online here: http://istics.net/stat/stat-text-booksfree-not-free/
Please see the edit by Tacio above.
Why are you using $\\chi^2$?  I'm at a bit of a loss to figure out how you are supposed to test the hypothesis $\\beta > 1.05\\, \\hatP\\{\\beta}$  This is a data-dependent hypothesis, and so isn't covered by Neyman-Pearson theory that I am aware of.  If you let $\\sigma_=o$ be the specified variance (uncertainty), then $\\frac{SSRes}{\\sigma_o^2} \\sim \\chi^2_{n-2}$ when $H_o:\\sigma = \\sigma_o$.
@whuber i didn't bother doing this as the other answer already answered this, i have a tendency of no posting something already been said by someone else. In this particular case, admittedly, i also forgot. I am using an ipad so i want to minimise latex writing. The other answer answers this correctly.
@whuber Well, in this case, I want to know 1) if "Since $X  \\sim N(0, \\sigma^2)$, we know that $X/\\sigma \\sim N(0,1)$. So $X^2/\\sigma^2 \\sim \\chi^2(1)$. But this means that $X^2 = \\sigma^2(X^2/\\sigma^2) \\sim \\chi^2(\\sigma^2)$." is correct. and 2) if the fact that I used the transformation $W_1 = X^2/(X^2 + Y^2)$ and $W_2 = X^2 + Y^2$ (and $U_1 = X + Y$  & $U_2 = X - Y$ for the second one) and used that to apply the theorem I mentioned, is correct. You don't need to check the details, just the approach that I used. I guess I will try to write my answers clearer/simpler from now on...
Do you care to provide a citation for the rule of thumb?
+1, and I vote darkness.  A Google search yields a number of references to "unsupervised regression", many of which are of the structural equation modeling/latent classes flavor.  From a brief review of these papers, I would personally tend to describe them as applying least squares (LS) and expectation maximization (EM) methods to unsupervised problems, rather than "unsupervised regression"
(+1) Your 2nd paragraph reminds me of a recent discussion (with Gelman, Kosara, Wickham, among others) about infographics *vs.* data visualization, e.g. [Information visualization” vs. “Statistical graphics](http://andrewgelman.com/2011/07/information-visualization-vs-statistical-graphics/), [Infovis, infographics, and data visualization: Where I’m coming from, and where I’d like to go](http://andrewgelman.com/2011/08/infovis-infographics-and-data-visualization-where-im-coming-from-and-where-id-like-to-go/) or [Statistical Graphics and Information Visualization](http://bit.ly/tlhGjk).
As an approach, EDA doesn't particularly try to be non-parametric: boxplots, halfspace depth& company are not devoid of assumptions. It's not about not having a model, it's about *showing* the deviation from the model. The whole EDA philosophy is distilled in this Tukey quote: "Numerical quantities focus on expected values, graphical summaries on unexpected values.".
@robermorales, Fair enough, but I think that seeing the "pure" version of the problem makes it easier to understand messier, real-world visualizations/problems.
@Michael Then I would simply write $a,b \\sim \\mathcal{N}(0,\\sigma^2)$. Wouldn't you also write $a,b \\in M$?
for the 2 scores per phase, are they taken at different time or just repeated measurement (i.e. the second one is taken immediately after the first one)?
(c) regarding parallel: I was curious about the additional costs and effort in setting up both the algorithm to run in parallel and loading multiple instances; (d) any tutorials that you'd recommend?
@matt  That's a classic case for alpha-blending (see the image in the comment above).  It requires some experimentation with the level of transparency, but you'll likely be able to find something that works.  It still won't be perfectly clear how many points are at each spot on the axis, but it gives the viewer a sense of density that lines don't.
From page 2: *The estimator in (1.2) was intended as an intermediate step in the construction of the estimator of $\\Lambda$, not as a bona fide estimator of the hazard rate function.  However, it is tempting to view it as an 
estimator of $\\lambda$ itself; in fact, $\\hat\\lambda$ is frequently  referred to as "the Breslow estimate of $\\lambda$."*
$$ $$
Their result is:
 *For t such that $0 < S(t)  <  1$, as $n \\rightarrow \\infty$, 

$${\\hat \\lambda(t) \\over \\lambda(t)} \\mathop{\\rightarrow}^d (E_1 + E_2)^{-1}$$
where $E_1$ and $E_2$ are independent unit exponentials.*
I've changed your [tag:sample-size] tag to [tag:power-analysis], but I'm not 100% sure this is what you're trying to do. In power analysis, you can find the required sample size to reject a null hypothesis of zero effect given an effect of a size you would expect to find, but I'm not sure how you could find both the sample size AND the effect size without knowing one or the other.
Is the response variable "connections" a binary outcome (1 if connection occurs and 0 if not)?
Thanks a lot, I'm looking into this class of models and the `lme4` package as I'm unfamiliar with them, and I'll report my findings.
+1 I love this technique and actually submitted a paper on it to a risk analysis journal a few years ago.  (Some risk analysts are quite interested in interval-valued data.)  It was rejected as being "too mathematical" for their audience.  :-(.  One tip: when using numerical methods, it's always a good idea to supply good starting values for the solution.  Consider applying OLS to the raw data to obtain those values, then "polish" them with the numerical optimizer.
@SaR In this case, you may be interested in this related thread: [How to compute correlation between/within groups of variables?](http://stats.stackexchange.com/q/9918/930)
"***irrespective*** *of the properties of the different m starting distributions*" is inaccurate. While it holds for a wide variety of situations, several properties must hold for the CLT to work. The standard error of a mean doesn't rely on the CLT at all; you simply use properties of variances.
see also the first diagram [here](http://en.wikipedia.org/wiki/Simpson%27s_paradox)
this is an incredibly unclear response. Please try to write in English.
Taal: I just want to answer the question put forward. "How would you calculate an estimate of the origin country from one of the names?" That is, we draw a name at random and want to find out the most probable origin country for that name. For me it is impossible to calculate an estimate of the origin country without weights.
It was good that you displayed the graph.  "My graph clearly shows that there is an interaction...."  If that's clear evidence for an interaction, think about how similar the slopes would have to be for it to be "unclear." Let alone for clear evidence of no interaction.
I think you mean bar charts rather than histograms
It would be very nice to see your picture
many thanks!!!!
That's right.  The connection can perhaps be seen more clearly by considering Sqrt(M) = (b-c) / Sqrt(b+c).  Approximating the variance of b as b and the variance of c as c (as is usual with counted data), we see that Sqrt(M) looks like an approximately normal variate (b-c) divided by its standard deviation: in other words, it looks like a *standard* normal variate.  In fact, we could conduct an equivalent test by referring Sqrt(M) to a table of the standard normal distribution.  Squaring it effectively makes the test symmetric two-tailed.  Obviously this breaks down if either b or c is small.
Your response variable (college acceptance) is non-normal (specifically, Bernoulli). When the response is normal, the population mean & the average of individuals are the same, but this is not typically true w/ non-normal responses. To understand this issue, it may help to read my answer here: [Difference between generalized linear models & generalized linear mixed models](http://stats.stackexchange.com/questions/32419//32421#32421).