tttjjj/elt_bench huggingface dataset:
Good. Just be careful at writing down the corresponding likelihood function. With large samples the inferences will be similar (Bernstein–von Mises theorem).
If the data is completely absent at four time points, I don't think imputation is going to help. Multiple imputation helps when *some* observations are missing. But if there's nothing special about those four years, other than that data was not recorded in them, leaving them out of the analysis would be okay.
Yup, these are very similar problems - I'd be almost as happy if there's an answer (i.e. a way to obtain exact confidence intervals) in the multinomial case, as what I'm working with will generally have $n$ very small compared to $N$ anyway,
@user2168 A subtlety is that your winnings are unlikely to be *exactly* this formula.  You really have to take the expectation.  But, as the Central Limit Theorem shows, when N is sufficiently large, the number of wins is highly likely to be within O(Sqrt(N)) of pN.  This chance of a relatively small deviation from the expected number of wins does not alter the optimal fraction.
A bit more information about your data would be welcome. What is the result of each "trial" and how do you know (or on what basis do you assume) that the residuals are correlated?
@zihuang Close.  "Tangent space" is a little off the mark, I think: you can work entirely within the $L^1$ space of integrable functions.  The probability laws on the original space are complete in the sense that any measurable function orthogonal to them must be zero (in the $L^1$ sense).  They do not extend to a complete set of distributions for the product space, essentially because there is a proper subspace of $L^1$ functions on the product whose projections onto the coordinates *are* complete.
Does the needed number of samples at $t$ depend somewhat on whether the values of a time series are functions of previous values in the time series?
I won't provide an answer here, because the question will be moved. But you can try a few things to understand what's going on: 1. run lm( Y ~x1 + x2 - 1). the "-1" will remove the intercept. 2. use relevel to change the reference category of x2.
Let $X$ be a random variable taking values in $\\{-1,0,1\\}$ with equal probability and let $Y=X^2$. The correlation between $X$ and $Y$ is $\\rho_{X,Y}=0$, but clearly they are related: $Y$ is a function of $X$.
I don't know Stata, but it seems that the test of `Categ_1` is non-significant, and the other two, having negative values for the `chi2`, must not "meet asymptotic assumptions". Thus, I'm not sure that you can validly conclude that "the data do not meet the IIA assumption". Possibly nothing can be concluded relative to the IIA assumption. Instead, I think you want to find out what the asymptotic assumptions of the *Hausman test* are, and what it is about your data that violate them.
Do you already have the equation of the trend curve or does adding it include computing its equation from the data?
You can run linear regression with binary outcomes if you like. (Though I'd recommend ensuring that `Gender` is indeed binary, and not stored as a factor.) But why do you want to do this? In your example the `Gender` coefficient in `glm(RT~Predictability+Length+Gender, family=binomial)` tells you about the (log) ratio of odds of RT, comparing Genders, in observations among which Predictability and Length have the same values. There's no need to adjust or rezidualize the Gender variable before doing logistic regression
@Patrick you can definitely do that. But since I am taking log of the mean I prefer normal disp effect. A log normal distribution is another way to modeling distributions similar to gamma distribution. HTH.
"*because it will be significant if i have a larger sample size*" - with a continuous outcome, there will always be *some* estimated non-zero effect size ... so you could *always* come to the same conclusion as you just did. Does an argument that says "my hypothesis is always supported" make sense? If so, why bother with any sample at all?
Since the covariance matrices of the other datasets, e.g. dataset 1, have only values on the diagonal they are independent so I guess it is safe to assume that that is the case for all artificial datasets mentioned.
+1 It's nice to see not only a better reply come along after almost two years, but a reply that provides more useful detail than the references elsewhere in this thread.
From what I can tell, this sounds like an appropriate set up for a statistical experiment. I think your results will only be generalizable to populations with the same C variable (for example if C was male, then your results would only be applicable to males).
What is the model you use in R ? Does it include an intercept ? The second formula is the correct one when there's no interecept.
What you want to do largely can't be done, especially with so few data. That's pretty much it. Regarding your point that you don't have a convention like a threshold for p-values, you should probably be thankful.
Have you tried to work out the algebra involved?
Randomness is more than equal prior chances.  For example, one team in Spain could have been randomly selected.  Assuming equal team sizes, this gives all soccer players the same chance of being included in the sample, but it's a stretch to assume that a single team is truly representative of all players in the country.
It's worth noting that all three models are multiplicative in the sense that an increase in a regressor is associated with a relative change in the typical response. For the two log-linear GLMs, "typical" means arithmetic mean, while for the log-transformed LM we are talking about geometric means. Thus, the way you want to interpret effects and predictions is also a driving factor for the model choice, not only having perfect residual plots (these are data driven anyway).
@Sonia do you have it stored as a sparse `Matrix`?  Try limiting the number of singular values you compute... maybe just look at the top 10?
Just checking to see if someone can use R, Matlab or other tool to confirm my results. Thanks.
@Affine That's what is being done now.  I'm wondering if there's any way to interpolate the load at a more granular level to get better correlation.
@steffen, mzuba, yes, the partial derivatives with respect to both x and y are non-negative.
I should make it clear in the documentation but `calibrate` doesn't understand time-dependent covariates.  As Therneau has stated frequently, estimation of survival probabilities in the presence of time-dependent covariates is not a simple thing to conceptualize.
@RobertF not really enough to do so, and anyway if I were to go that route I might use a tobit model.
Hey Tom, thanks a lot for the paper. Yes this is exactly what I was looking for :)
Plotting data _is_ a manual check.
Please specify the context. If it's 'R', say it's 'R'. Update the title if possible.
It's possibly also worth saying that I have about 100 features for each user.
@gung thank you
@Glen_b I presume I do, although in my software this is referred to as 'alpha' > 1? A factor to which you multiply the mean, to get the variation.
As my econometrics instructor used to say, "This is a free country [USA -- StasK], you can do whatever you want. The *real* question is, can you defend it, and can you publish it?". AIC assumes i.i.d. data, which is clearly violated with `lmer`-type mixed models. Different parameters have different information content, and it is difficult to quantify properly when you have random slopes.
But if missingness is related to X1 or X2 then surely it is good to have two separate models - since they will be incorporating that information.  That is to say, when in future I get a missing X3 I will know to be biased in the correct direction.
Do I understand correctly that you have data on your outcomes for the same participants before & after the intervention? If so, there is a sense in which you do have 2 groups, & you can run a paired / dependent / repeated-measures t-test (different names for the same thing). However, I have no idea what it means to say that you have quantitative outcome measures that are "non-parametric". Can you clarify this?
Heteroskedasticity is present, no linear relationship (partial residual plot), and non normal error terms.
Corollary: look whether someone coded a missing value as "9999" instead of "NA". If your software uses this value at face value, it will mess up your analysis.
IMO your question is a good fit for CV, but since the answer to your question is not really statistical (you already know how to do it for OLS) I will just re-direct you to the [lqmm](http://cran.r-project.org/web/packages/lqmm/index.html) and the [robustlmm](http://cran.r-project.org/web/packages/robustlmm/index.html) packages (and the citations therein).
It's not clear to me either what your text book means here.  Could you add a bit more detail to your question - perhaps a longer quote from the book?
Very nice answer. Could you briefly say how to apply this framework to the example I give in the question? Does it give the expected answer (A)?
You did not say "multivariate" normal, so I added the "independent" in parentheses (i) to fit within the context of the question and (ii) to remove the possibility that someone would point out that, as stated, your claim was incorrect. :-)
There's often confusion about the (primary) use of cross-validation to validate a model-selection procedure for a particular data-set, & its (secondary) use as part of a model-selection procedure. In this case you're cross-validating the procedure that gave you *one* set of parameter estimates, not picking the best of the ten sets - they're disposable. If you were to do the latter, *that* procedure would itself need to be validated. Dikran's explanation [here](http://stats.stackexchange.com/questions/27750/feature-selection-and-cross-validation) is good & clear.
In the context of a panel dataset, a dummy variable for each unit.
You are confusing probability and probability density. Probability density can be greater than 1. The assumption that you should replace probability density by cumulative probability for maximum likelihood is quite fallacious.
Thanks very much for this - I'll look into the paired Wilcoxon test
You can create a dummy that's 1 for $t=48$ and 0 at all other periods. Then re-estimate the model. That will keep this outlier from skewing up the forecast. If that's not what you have in mind, you should elaborate on the second paragraph.
Can you get user feedback? Usually, with clustering approaches, it's about enabling the user to learn something about the data; so user feedback is the best source of validation data.
See [Where do the assumptions for linear regression come from?](http://stats.stackexchange.com/q/55113/32036)
Okay, but then don't two Gaussians with diagonal covariance do the trick (making sure that the ordering of the eigenvalues matches up, and then scaling the "rings")?
http://stats.stackexchange.com/search?q=error%2Bpropagation
More complicated than I would have liked but thems the breaks.
@Glen_b In case of Multivariate Gaussian, it's nice how the marginals end up being Gaussian too. Similarly i was hoping that jointly log-normal would imply that all the marginals are log normal and together they have some multivariate distribution. To be honest i am interested in finding out what this multivariate distribution ought to be or if there are infinite possibilities to choose from what would be the simplest choice so when the logs of the marginals are taken we get Gaussian marginals and a multivariate joint Gaussian density. I look forward to your thoughts.
Cesare, I think you're confusing homo- with heteroscedasticity.
Consider something more modern than k-means from the 60s. If you want to do time series clustering, read up on time series clustering methods!
Replace the nodes of your graph with $Heat_t$, $PlateArea_t$, and $Friction_t$, where $t$ is the time step (0...N). The graph is now acyclic: $Heat_t$ points to $PlateArea_{t+1}$, $PlateArea_t$ points to $Friction_{t+1}$, and so on. The trick that makes this infinitely long network possibly tractable is the assumption that the parameters are the same across time, so all the lines connecting $Heat$ nodes to $PlateArea$ nodes have the same params (and so forth). [Page 430 of this](http://www.stat.cmu.edu/~cshalizi/uADA/12/lectures/ch21.pdf) gives a good diagram for such a graph.
@user862, the proofs I know off-hand aren't particularly direct. Durrett has a derivation in his probability book which is pretty clean. It takes 3-4 pages, I think; which, if you've read any of his books, is a long proof by his standards. Resnick takes a Little bit more abstract approach in his stochastic processes text. Constructing and wielding bigger hammers lets him get more general results, though. Ross undoubtedly has a treatment in his stochastic processes book, but I'm not that familiar with it.
I am interested too. For the exponential smoothing, I usually use grid search which can be long for triple exponential smoothing.
Mann-Whitney (comparing two groups) is only the special case of Kruskal-Wallis (more than two groups), so yes.
Welcome to this site.  A search at the upper right for "compare logistic" comes up with multiple threads that should help you.  But I'd also like to ask what you have tried so far.
Not a good recommendation for extremely small sample sizes. With 7 and 2 samples, the U-test will fail, no matter how large the difference between the mean of the groups. Look at my answer for an example.
You might be putting the cart before the horse.  With such a small sample size, you might not even need statistics to make your point about observed effects.  Statistics and significance tests are best used for extracting counterintuitive signals from noisy data with complex structures.  In your case, with two time periods and 5-8 people, you could make you point if you just plot the data and write about it.
Is the output (or response), which I assume is opinion, a numerical variable or is it also categorical?
That's right.  The function find.cutoff(prob) gives you a point at which there is a a given probability that the observation was drawn from the lower component, i.e. it returns the x such that Pr[X came from lower component | X=x] = prob.
The title was changed by Emre.  Honestly, what's more important is 'Clustering groups'.  Based on frequency & date would be nice to have -:)  Thanks.
+1 Interestingly, when you calculate the $(1,4)$ entry in $D^8$ by diagonalizing it, the resulting formula is *identical* to the one produced by the "easiest way": the eigenvalues are $1$, $0.7$, $0.9$, and $0.6$ and the signs $1,-1,-1,1$ come from the last row (or column) in the inverse of the transition matrix (matrix of eigenvectors).
@January Perhaps I'm not using the right language here. I see AIC as penalizing the log-likelihood by the number of parameters. That's why I said "adjust". Feel free to edit the question for clarification.
Thank you for your reply.
In the original paper, one of the parameters that is estimated is $\\sigma_{a}$, the variance of the technology shock. Unfortunately, in Dynare, while you are able to estimate transforms of parameters, you can't do so for the standard error of shocks, such as the technology shock in the model equation. Instead, you must assign a prior to the standard error of the shock directly. Given that the prior of the variance is inverse gamma I assumed that a suitable prior for the stderr would be the "square root" of this invgamma distribution. Does that make sense?
Ariel is asking whether there are a wide range of scores from tests (e.g. out of 100) or just a pass/fail result from a test. It also might be wise to have a control group by having another group take the two tests without the module (as some people learn just from tests).
A similar question was asked [before](http://stats.stackexchange.com/questions/17121/ordered-logit-in-jags). It seems to be a problem with the initial values. You seem not to user-specify them, which you should to avoid problems.
Nice answer. Maybe one thing to add would be the A-optimality criterion, which is the trace of the var-cov matrix, so here we are minimizing the sum of the variances. This goes a bit in the direction of what the OP was asking about.
"*I'm writing a program to help me gamble on cricket*" - if you make money on it, do the best answers get a cut?
@whuber I do Bayesian reasoning here: I don't simulate outcomes of the rolls of die - for that I'd use categorial distribution. I simulate the probabilities with which the die rolls, given the already observed historical outcomes, i.e. the posterior distribution of the parameters $\\pi_{1\\ldots6}$ that define the die.
Thank you I found your answer really useful. Regards Cody
always these bayesian guys...
Hello, welcome to CV! Answers are better when they are standalone, meaning that a reader can understand the answer without referring to an external link. (Also, external pages can change or move.) Would you mind expanding your answer a little bit?
@Joel W.  What do you mean when you say the sample is biased? Is it the estimate of the mean based on the sample that is biased?  Any sample estimate is going to differ from the true mean and some can be far off.  When sampling at random this is due to variance not bias.  It is not right to say a sample is biased because the distribution of the sample is known to look a lot different from the distribution for the population. In small samples many can look unrepresentative for one reason or another but random sampling is not biased sampling.
I think you should detail the context of your question. Tell us more about the NNT and the variables, explain what is the classification about... This will be really helpful in answering your question.
Thanks for this excellent answer and the help!
Unless you know of some underlying structure to the component processes, I think you are going to have to check each one to see if it is Poisson. As an example of a useful structure where you could avoid this, if you knew the superposition process was Poisson and the sub-processes were defined by a splitting mechanism, then you may be able to immediately know the sub-processes are also Poisson. Look up splitting a Poisson process if this is unfamiliar.
@whuber Thank you so much!
A better approach would be to calculate the **residuals** (i.e. `e = actual - predicted`) and then plot those on the y-axis and the **predictor** on the x-axis.
I'm not R user and can't speak about homals(), but I doubt that HOMALS analysis quantifies missing data. It doesn't make any kind of imputation, as far as I'm aware. In HOMALS, there exist such a thing as treating missings as "passive data" while computing the components, but is not imputation or quantifying missings.
Let's see. When $n=5$, the sequences of 0 (not falling) and 1 (falling) in which two falls are in a row occur are 00011, 00110, 01100, 11000 (for total chances of $4p^2(1-p)^3$), 00111, 01011, 01101, 01110, 10011, 10110, 11001, 11010, 11100 (for total chances of $9p^3(1-p)^2$), 01111, 10111, 11011, 11101, 11110 (for total chances of $5p^4(1-p)$) and 11111 (a chance of $p^5$).  The total chance works out to $4 p^2 - 3 p^3 - p^4 + p^5,$ which differs from  $(n-1)p^2 - (n-2)p^3$ by $p^5-p^4\\ne 0.$ For $n=1000$ and $p=0.00001$ the error is almost undetectable, but the formula is still incorrect.
When you're avoiding red-green-yellow distinctions to accommodate the approximately 6% of western males (and 1% of western females) with some form of [red-green color blindness](http://en.wikipedia.org/wiki/Color_blindness#Red.E2.80.93green_color_blindness), there are not too many other choices left!  (I prefer blue-yellow scales to blue-red scales, but often shades of yellow do not render reliably or reproduce well.)
Markus, the first paragraph belongs as a separate question rather than within an answer.  The subsequent material looks like a good observation but it is hard to follow without being told what $A, B,$ and $C$ represent.  Please bear in mind that different users will see the answers in different sequences, according to their preferences and when the answers were last edited, so each answer has to be readable independently of the others (although of course you can link to other answers).
You need to explain what you want to learn from the factor analysis or why you are considering it. It's not an end in itself, you don't *have* to do it, at any level.
I wouldn't call it trial-and-error. The idea is not to find values that happen to work but rather how dependent you are on your assumptions - and the proper interpretation is that if there's a large difference your results will be unreliable for both values you tested.
if the probability of winning any of the cash prizes is independent, then there is no way that these probabilities can be correct. They can only add to 1, so this question is wrong.
Welcome to the site, @jigbaja. This isn't really an answer to the OP's question. It is more of a comment. Please only use the "Your Answer" field to provide answers. I recognize it's frustrating, but you will be able to comment anywhere when your reputation >50. Alternatively, you could try to expand it to make it more of an answer. Since you are new here, you may want to read our [tour page](http://stats.stackexchange.com/about), which contains information for new users.
+1, I second all your recommendations and have only one *very* minor quibble: there is no "c" in Frank Harrell's name... He's American and not French, after all ;-) I'll wait for another 317 rep and edit your answer...
With nonlinear problems, you likely need independence rather than uncorrelatedness. My gut feeling is that you will be exactly identified, accounting also for nonlinearity. If your second stage is nonlinear as well through the network action, then God only knows how that works in terms of instrumenting and identification.
And a paper by Hall, Tavlas and Swamy (2011) [The Non-existence of Instrumental Variables](http://www.le.ac.uk/economics/research/RePEc/lec/leecon/dp09-16.pdf).
I compared the dput object with the table in the OP and concluded D1 meant "old frequency" and D2 meant "new frequency" but had forgotten I'd made that assumption by the time I wrote the post.
It depends on what you want to do with the SVD. If you still feed in the SVD-reduced matrix $\\mathbf A$ into your algorithm, the effort remains the same, but results might be better as you concentrate on directions with maximum variance (and hopefully filtered out noise). However, it is also possible to directly put features of the decomposition into your algorithm (this, however, means that you need to apply a similar decomposition for each prediction). **If reduction of the algorithmic effort is what you are aiming for, I would rather look for a dual version of your algorithm**
"Goodness of fit" depends on how the fit was performed. For instance, the appropriate GoF measure for a maximum likelihood estimator ought to differ from the GoF measure for a least squares estimator when the random component is not an additive Normal variable. If you have a specific problem in mind, then you might therefore wish to indicate how you performed the fitting.
It's also worth searching online for sites featuring bad graphs. Very informative.
Hi Yuto, welcome to CrossValidated. Routine bookwork questions can be posted, but they are treated differently. See the link in Nick's comment above - please read it carefully - and then modify your question as appropriate (that is, we're not going to tell you directly how to do this - if you explain what you've tried and where your specific difficulties lie we will be able to give you some hints and suggestions. As Nick says, you fit regression with `lm`, but if you didn't know even that, CV will not be a suitable place for you to learn about basic use of R.
@RomanLuštrik, I've fixed the code and added the correct output. Feel free to remove the correction.
Angrist and Pischke's [Mostly Harmless Econometrics](http://www.mostlyharmlesseconometrics.com/) deals with the question to some extent.