tttjjj/elt_bench huggingface dataset:
I am interested in the last statement: "the very first commercially available (and working) Bankruptcy model implemented by the credit bureaus was created through a plain old linear regression model targeting a 0-1 outcome". Which model was it? I believe that the first model was RiskCalc by Moody's, and even the first version was a logistic regression model. The developers of that model were not CS people with a background in ML, but rather in econometrics.
Why not? PCA is not an inferential analysis, it is just reduction/descriptive analysis, thereby putting no assumptions about the shape of distribution.
+1.  The explanation in ESL is great, and the associated diagrams help a lot.
Say that you take the derivative with respect to $p_1$ The derivative "can enter the summation", by linearity. Now, $p_1$ appears in the first term ($i = 1$) only. The derivative of all other terms ($i \\neq 1$) are thus $0$. This is why the summation sign disappears.
What are you using to look for a "multicollinearity problem?"
Research problem in optimization.
So does there exist fast, message-passing algorithms for inference in these kind of un-triangulated graphs or do you have to resort to something like Gibbs sampling?
@R Learner: so essentially you're saying that my answer appears to have been correct? In that case, if nobody else manages to come up with an even better answer within a reasonable period of time, please remember to eventually mark it as "accepted".
Why not just plot the three ROC curves for -1 vs. 0/1, 0 vs. -1/1, and 1 vs. -1/0?
To compare (statistically) a part with a whole is to compare that part with the rest portion of the whole.
@slhck There are other solutions.  For instance, you could construe the responses as rounded versions of continuous values.  If the rounding is to the nearest integer, use $\\sqrt{1/12}$ in place of the standard deviation when all the values are the same.  (This also has its problems, but it at least acknowledges the potential for sample variation in the mean response.)
Are there any natural constraints in your DVs?  For example would there be something like $DV1 + DV2>DV7$.  Taking these into account may be useful for your analysis.
@ Tim: I assume that you are the beginner in applied time series econometrics. So, I recommend you to have a book Applied Econometric Times Series by [Walter and Enders](http://www.amazon.com/Applied-Econometric-Series-Probability-Statistics/dp/0470505397).
What is your reason for choosing a prior that doesn't reflect your prior beliefs? That seems to contradict the reason that you use priors in the first place...
ok, so I still don't know the answer but I had an idea for an approach - let me know what you think.

I know that each model is binomial, therefore if I make a measurement of the fraction $f=0.3\\pm0.2$, I could calculate the probability that my measurement is consistent with the other model, ie the $f=0.15$ model. I suspect this can then be generalized to my measurement being any value, and calculating the probability that either of the models is correct and looking at the difference in the quality of the fits. I could then say 'our findings support model 1 over model 2 at the $X\\sigma$ level'
(I just added the self-study tag myself; couldn't do so before since other edits were pending.)
so I did initially misunderstood the answers to previous questions.... thank you so much :)
I can obtain a misclassification rate of just 10% with these data, sight unseen, by always predicting 0!  Surely *any* method you try will improve on this, right?  Perhaps some more information about the quality of your results would help us understand the situation better.
We would usually call such variables 'count data'. Models with predictors would generally be fitted using generalized linear models. Depending on the situation, any of several different models might be used.
You need to do panel-data econometrics while being "new" to econometrics? Panel-data econometrics is a complicated if not advanced sub-field, and still not consolidated enough. Too many set-ups, too many estimators, too many results and different sets of estimator properties. To what degree your master thesis will be seen as an "educational endeavor" (permitting to make easily all sorts of simplifying assumptions), and to what degree it should try to satisfy the standards of an actual research-level paper (requiring that you properly justify your simplifying assumptions)?
I think it's interesting that whether a fixed or random effects of subject is fit, it seems to extract the same amount of error variance. Don't you?

Naturally, I agree that a random effect of subject is the correct way to run model 2. I'm still curious about model 1, though, given that most of the (older) members of my department use repeated-measures ANOVA for everything...
It comes from the logistic function (http://en.wikipedia.org/wiki/Logistic_function)
@ martin I used Weibull distribution by default. Could you explain how different types of distribution in my population contribute to the analysis of the model?
The right type of regression depends on the *dependent* variable, not the independent ones. Please clarify what your model is, including DV, IVs, research question etc.
I think that should be included and since expected frequency is less than 5, then it should be combined and create a common  category (30,34]
PS -- the real reason I included the code is to prompt responses from anyone in the programming realm.  This function is probably found (and more elegantly) in some 'VBA stats snippets' resource ... I should search on that.
But the `lmer` results have higher standard errors than `lme` - opposite of what you wrote? Anyway, the differences are so big that one of the results must be wrong, OK?
I hope you have a *really* big sample! This is going to be hard to test.
Excellent answer! thanks
You can omit one of the variables and still obtain an unbiased estimate of the other if they are independent.
As this question appears to be based solely on coding, you will likely have better luck if you move this question to [Stack Overflow](http://stackoverflow.com/). Best of luck!
This would require a book (or most of a book) to answer.
No that's not correct. You might want to review how to set up convolution integrals - not just the limits, but also the integrand.
First, for the sake of full disclosure, I do not understand everything in both articles. Though, thank you very much for sharing them (especially the second one). They were very interesting. From what I took from the first article, in the neural network we only have one instant to kick off the cyclic behavior. However, the network was trained using an entire series, which it re-launches into once it is given the input. The recurrent network's feedback then causes the network to continue to cycle.
It sounds like when you say you have "one sample" you mean that the sample you have has only one observation. Is that correct? The terminology can get ambiguous, but usually when someone refers to a sample, they are referring to a *collection* of observations (usually modeled as realizations from random variables).
Andy thanks! I didn't know about the SUM command but it looks like that will do exactly what I want. The dataset is perfectly structured for this as the variables are all in order and have a 1 as a punch.
Javid, when you quote part of the question in your answer, you might consider using the markup intended to indicate a block quotation (`> ` at the start of the quoted block) rather than just bolding it.
--> http://www.jstor.org/discover/10.2307/2685604?uid=3737592&uid=2129&uid=2&uid=70&uid=4&sid=21101484916817
@Uri, thanks, hadn't thought about that.
You will need to give a lot more detail on what you are trying to do and what kind of seasonal adjustments you are considering.
An ROC curve shows the TPR as a function of FPR. Neither of these measures exists in the context of regression, so there is no such thing as ROC curves for regression.
Thank you for your answer and the linke. In the meanwhile I continued my research and I found quite an easy way that might provide a good model. I will post this as an "answer" by myself (then people will more probably see it and discuss it).
@Wilmerton The goal of creating Pearson residuals is to create standardized residuals that should be mean zero with variance of 1 if the model specified is true. I hope that I have been able to clear up some of your confusion.
This is a problem with stepwise regression and categorical variables.  See this (very) recent question for more information:

[How should I handle categorical variables with multiple levels when doing backward elimination?](http://stats.stackexchange.com/questions/6050/how-should-i-handle-categorical-variables-with-multiple-levels-when-doing-backwar)
Are all of the steps Gibbs steps?  If so, why not just write your own loop and forget about Rjags.
This is a great answer and exactly the kind I was hoping for. However, I want to point out for other potential answerers that a good example need not concern an inference that was made by researchers/statisticians (and, in particular, not only those using the best available methods). Rather, an equally good example might describe a case in which the media, the public, or some other group drew an invalid causal inference from correlative evidence (as long as this incorrect inference had notable effects).
Beside possible lack of power (but power/sample size should have been determined beforehand), what else would you like to say? If you cannot highlight an effect of your factors on *this sample*, that's all what you can say, IMO (keeping in mind that you can't accept the null).
I like it because it makes no assumption on the dependence between the variables in the sequence. It also provides a nice clean basis for thinking about things like Riemann sums of normal random variables (Think: Integrals of Brownian motion), allowing one to conclude the distribution is normal without having to think very hard or carefully about anything strange that could be happening in the limit.
It's great to see you here, @RussLenth. I've read your "Some Practical Guidelines for Effective Sample Size Determination" many times and find it very useful.
The maximum a posteriori estimator is equivalent to a penalized maximum likelihood estimator when the penalty is chosen to be equal to the log-prior. The penalized MLE can be thought of as the MLE under a constrained model with $\\tilde{m}$ number of parameters. The AIC is then proportional to $ \\tilde{L} - \\tilde{m} $, where $\\tilde{L}$ is the maximized penalized likelihood.  In this case, the challenge is to approximate $\\tilde{m}$ - [see this paper](http://www.ncbi.nlm.nih.gov/pmc/articles/PMC2629611/),  section "Information based inference for penalized likelihood estimates".
If you have nothing to do with the vaccine, I guess you will have to rely on studies of its effectiveness from elsewhere. 

The data may tell you about the prevalence of the disease in the country. You may want to consider sampling issues.

The data may also help you model QALY if you can find any models which go down to the level of looking at things like interaction between diet and vaccination.
Thank you @mpiktas. I have now updated that.
The two numerators are equivalent (by one definition of conditional probability) and the denominators are equivalent, too (by the law of total probability). Thus the "definitions" (which aren't really definitions; they are just two formulas for the same thing) are just different ways to perform the calculations.
It is unclear what your *real* question is.  It is obvious that no function can look like $\\chi^2(1)$ (which is an exponential distribution) and still essentially be supported on $[0,1]$, so there's a question about your characterization.  Do you need to determine the null distribution of a test statistic?  If so, we need to know how that statistic is computed and how the data are generated.  Otherwise, what does this "function" really represent and how do you intend to use it?
I suggest you to read Lecun, Efficient Backprop, 1986 (http://scholar.google.it/scholar?cluster=15983004533596008350&hl=en&as_sdt=0,5) where the author proposes and discuss some tricks and tuning about NNs. I always normalize input features between -1 and 1.
Discrete random variables aren't normal. What exactly are you trying to achieve? The same means? The same means and standard deviations? Something else?
That is helpful, but yes, for example, what is the smallest number of birds in a flock/alerts by subspecies (ssp) and part of study site (ssp.zone)?
I am agree with gui11aume. I just wanted to suggest that instead of using abstract centroids you may want to use one or more exact typical organizations for every class. In this case it is more classification with given samples, but if you have any expert bias it should work better.
@RyanSimmons: You're welcome!
@JakeWestfall - Yes, this is distinguished from linear in time (or other predictors). So a quadratic growth function is nonlinear in time but linear in its quadratic parameter(s). As I understand it, this constraint is required for the average of the individual models to be equal to the model of the average.
For record keeping: The DOI of the "The distribution of product of independent beta random variables with application to multivariate analysis" is 10.1007/BF02480942.
I know I need to include a random effect, I just meant I have to think more about random intercept or if I should include a random slope for anything, random on a, b, or both ((log(tl)-1) |tank or ~1|tank, etc.).
How are you assessing distributional shape? Some kind of diagnostic display?
Check the definition of the [mean squarred error](http://en.wikipedia.org/wiki/Mean_squared_error#Definition_and_basic_properties) and correct your computations.
I am actually not really sure whether the data can be considered as a random sample. The observed values (social network) are somehow random in nature, yes (although I cannot easily quantify the error - binomial confidence intervals would just be https://en.wikipedia.org/wiki/Binomial_proportion_confidence_interval#Normal_approximation_interval, right?). But the expected values (authoritative population data) are register data and definitely not coming from a sample. Is a GOF test in this case not even necessary?
Thanks whuber! The question has been updated with further clarification details.
What is the aim of the analysis? What question(s) are you trying to answer?
I managed to compute the multiplicative indices since it's simply the division of a vector by a scalar. What now bothers me is that these indices are too small by a factor of 10 .. but afterwards they appear to be reasonable seasonal initial indices.
Great explanation but slightly confusing: the way you describe the first paragraph, isn't that KL(Q||P)?
+1 I would use *credibility region* instead of *credibility interval*.
+1 For recognizing that regressing $Y$ on $X$ requires errors-in-variables and that the answer really requires understanding the objectives of the analysis.
I mean if for the semi supervised we assume that we have no idea about the label (observed value) for the majority (900 records). So if I try to find the best cut point by testing the algorithm on those records it is actually won't be a semi-supervised algorithm, am I right?
This gives nice, and useful, intuition, but should not be misunderstood as being a formal proof. That said, an actual proof (as in @mpiktas's answer) utilizes machinery with which the OP is probably unfamiliar.
Glad to hear it and thanks for saying so!
Do you want to write it up as an answer?
See [Machine Learning with R](http://books.google.ca/books?id=ZQu8AQAAQBAJ&printsec=frontcover&dq=machine+learning+with+r+brett+lantz&hl=en&sa=X&ei=UWh4U-XwMsGxyASXtYGICQ&ved=0CDgQ6AEwAA#v=onepage&q=machine%20learning%20with%20r%20brett%20lantz&f=false) by Brett Lantz.
Can you tell us more about what you are trying to do? [this post](http://www.statisticalanalysisconsulting.com/how-to-ask-a-statistics-question/) on my blog may help
If you take differences, you treat the variables as being at least interval scaled. So, if the sample size is not too small, you could as well consider a linear (mixed effects) regression.
It is the variance.
@GlendonTrullinger, Glad that I could be of help. Sorry for the typo in the first draft.
I think that the question raised is so broad... And perhaps you should make it more specific. One notion, however. Each clustering algorithm has its objective function, the goal; and it is how well it pursues its goal is the criterion. Goals are different because various definitions of what is cluster and what is density exist.
This question was answered by Prof Hyndman.                    Theta method does not handle seasonality.
My intuition is it's not possible because the covariance of the response indicators is not observed. But I might be wrong.
What are your questions of interest? What exactly do you want to find out?
Added some more info! The prior is vague but not flat. It has some influence as it regularizes the estimates and pull them slightly towards 0.0 which is what I believe you want in this case.
(+1) Your second point, in particular, is an important one and, once understand, helps elucidate the differences in the two concepts involved.
Perhaps you could provide some example data?  From what your data looks like in my head, you may need to just run two totally separate regressions.
@Glen_b Here you go http://www.scielo.cl/pdf/ijmorphol/v29n4/art60.pdf !
Thank you so much for your help. I am going to explore your leads
Tricky, as you will be the one assigning numbers to pain levels, and there is nothing to prevent you changing your mind over time...however, it is worth doing. My mother had diabetes, my father wrote her sugar levels and did some fairly simple statistics, just average with a moving window, really. The doctor was impressed and adjusted the medicine. I keep my cholesterol numbers in a chart; trends are quite clear, improvement from the diet and fish oil pills has been dramatic. Well, good luck.
+1, your point about differences, even for linear models, w/ model misspecification is a nice one. A small worked example illustrating this would be a really great addition, should you be interested in providing one.
It's a great question and presents a real challenge.  You might be heartened to know that many people have had to confront problems like this one.  Check out the Modifiable Areal Unit Problem (MAUP) and look into [dasymetric mapping](http://astro.temple.edu/~jmennis/research/dasymetric/index.htm) (which has two meanings: this is the one concerned with changing geographic regions for the purpose of analyzing underlying data).
Other reactions to the course are actually [pretty negative](http://www.angrymath.com/2012/09/udacity-statistics-101.html), and I can understand why. I watched a few videos and was unimpressed.
I think a training case involves an input and labeled output that is used to back propagate. An input vector can be used as part of a training case, or it can be used as input to the trained neural network to get a resulting prediction. I don't understand the statement about them being perpendicular to one another.
@Nambari - welcome to the world of the "wise".  The beginning of knowledge is to know that you know nothing - to be a student.  I try to be always learning myself, and try to be correctable by anyone speaking truth.  If you play with the Eureqa tool, and try on appropriate sample data both every general form of "target expression" and every "error metric" then you will start to know this deep deep thing.  I don't have a good answer.  L'Hospital (aka L'Hopital) formulated the first least squared expression in 1696.  A good start is the use case - where the mind comes in.  What is that?
+1 yes, nothing is perfect. Tickdata.com (whose paper is mentioned) also includes outliers and they also strip out too much good data (when compared with another source). Olsen's data is close to being terrible, and I generally just indicative. There's a reason that banks pay big operations teams to work on this.
Please summarize the contents of the link to prevent link rot
Could you add details to your quite reserved question. Examples. And do you consider here both stages of LDA - extraction and classification, - or classification only?
So it's boiled down to (36^2/38) - (36/38)^2?
Thanks again for this valuable and thorough feedback! I will leave the question open for a few days to collect some more opinions
Love this one!!
Thanks! What do you mean by in sample and out of sample?
@Xi'an: Thanks! (1) It is nice to hear from another expert! I must confess I have been a fan of your for quite a while, and your books are still on my to-read list (once I get the money). (2) May I have some pointers to some references on this matter? Thanks!
+1, fabulous overview of ANNs.
For the first version of this project, I don't want to consider demographic or physical location.  What I want to look at is the transactional data for each customer.  For each customer, I have purchase history which includes the merchant where they made the purchase and amount.  I have grouped merchants into merchant types and I will be examining features that are counts of purchases at a merchant type over the life of the customer and dollar amount at a merchant type over the life of the customer.
+1 An impressive thoughtful response.  Welcome to CV, Nick!
+1 Thnx for your help, I'll take a closer look if I can find the derivation :)
Can you stick some dummy data in the front so I can get a better sense of what you are doing?
As long as your data has a frequency other than 1, seasonal ARIMA models will be considered. If a non-seasonal model is being returned, then either the seasonality is very weak or the data are not in a ts object with frequency > 1.
Correlation between the independent variables does not have to imply that the effects will be insignificant, even though we know that the higher the correlation, the higher the estimation uncertainty. See also the various links e.g. [here](http://stats.stackexchange.com/questions/95404/is-centering-a-valid-solution-for-multicollinearity/95420#95420) for a further discussion.
@Andy Thanks for pointing that. I realize afterwards that my first sentence does not sound well indeed. I'll update my answer, pls let me know if this ok with you.
@whuber Thank you for your commentary. I described the details. The difference between "handbook" and "textbook," as I see it, is in the depth of details and scope of topics. Handbooks in this case exclude discussions and explanation and concern upper-level topics usually omitted in textbooks.
Welcome to the site, @zhanxw. At present this is actually a comment asking for more info, not an answer. Since the OP has provided more info, would you expand this into a proper answer? You could also make it into a real answer by explaining how this information is needed to address the situation. Note that, if this cannot be turned into an actual answer, it may need to be deleted. Since you're new here, you might want to read our [FAQ](http://stats.stackexchange.com/faq), which discusses things like this.
boundaries in input space can be find via convex-hull or concaved-hull yet in kernel space there is no obvious way to find such exact boundaries as geometrical algorithms are not feasible.
Thanks. Just curious, and only if you know of the top of your head, do you know if it would be possible to specify a different statistic as criteria? I.e., rank according to Adjusted $R^2$ or AIC?
*K:* is the *Maximum order of Fourier [terms](http://en.wikipedia.org/wiki/Discrete-time_Fourier_transform)*. Alternatively, you could ask what the discrete time Fourier decomposition [is](http://www.math.ubc.ca/~feldman/m267/dft.pdf).