Talk:Generalized linear model

message: Whoever wrote out the "intuition section' deserves a medal. Such clarity I have rarely experienced in a wiki statistical article. +1 for making clear the relations between the Poisson and geometric distributions.
Theblindsage (talk) 23:45, 15 June 2015 (UTC)
thread_title:  Plaudits 

message: There are no errors here, but this article needs to be cleaned up and fleshed out a bit.  I will try to work on it soon... --shaile 00:33, 14 April 2006 (UTC)
replies: text: I don't like: "In statistics the generalized linear model (GLM) generalizes the ordinary least squares regression." Surely OLS is an estimation method, not a model? In my view, it should say "In statistics the generalized linear model (GLM) generalizes the linear model."  Blaise 21:45, 23 September 2006 (UTC)
replies: text:  You better believe OLS implies a model. Granted, a model that most would agree is wrong for their data, but some models are useful, so we use them. I'd suggest merging those two, they cover the exact same subject. Pdbailey 13:39, 26 September 2006 (UTC)
replies: text:  I agree with Blaise, though I must say the one he suggests does sound a bit like a tautology. Wherami (talk) 17:03, 30 September 2011 (UTC)

text:  I modified the sentence. As it turns out linear regression and least squares do not necessarily go together. Wherami (talk) 17:26, 30 September 2011 (UTC)
thread_title:  Cleanup 

message: The edit I made (that was rved) did trim the overall size of the page and reduce the number of examples, but as it stands it is very difficult to understand what a GLM is. The basic question is what is a glm? After we have said that, we need to say why you might want to use one and then I think we should get a little into how parameters are estimated. I'm going to rv it back and expand substantially on what I wrote last time.
It would be great if we could expand on the part about using any CDF. here we have the alternative example where Y = 1 if \eta > 0 and is zero otherwise. Pdbailey
: Would you explain what you meant by this sentence: "Because the variance portion is not constant, there is no sense to least squares, instead the parameters must be estimated with maximum likelihood or quasi maximum likelihood."  It is not clear at all...  Thanks! (Also, PLEASE sign your comments! All it takes is 4 ~'s)  --shaile 22:21, 20 April 2006 (UTC)
replies: text: Sorry, I don't have time to fix it up right now. The general idea is that OLS won't work because none of the assumptions hold (independance, constant variance)Pdbailey 00:11, 21 April 2006 (UTC)
thread_title:  proposed changes 

message: 1.  There is no mention of the probit link.  From a passage in McCullagh and Nelder, the probit work is historically important, in particular the presentation of the scoring algorithm in an appendix written by R.A. Fisher for a paper by the toxicologist Bliss.  
2.  Mention that in practice glm's provide an important way to address heteroscedasticity.  
My apologies in the event I have overlooked some passage that addresses these concerns. 
Dfarrar 04:48, 20 February 2007 (UTC)
thread_title: History and Motivation

message: Should the probit link be included in the table of canonical link functions, or is it not considered a canonical link function? Bill Jefferys (talk) 23:43, 20 December 2007 (UTC)
replies: text: It isn't a canonical link for any of the distributions there.    I do suspect there is a distribution that makes it such, but don't know off the top of my head what it might be.  Baccyak4H (Yak!) 02:23, 21 December 2007 (UTC)
thread_title: Probit link?

message: Shouldn't beta be in boldface throughout? At present some of the betas are bold and others aren't. This is particularly disturbing when it happens in Xβ.
--84.9.83.26 (talk) 11:46, 19 December 2007 (UTC)
replies: text: Yes, consistency is a good thing.  Nice catch.  Baccyak4H (Yak!) 14:47, 19 December 2007 (UTC)

text: Update I did that fix; sorry if I missed any.  I used wikimarkup (multiple single quotes) rather than html markup (<tags>) for consistency with the rest of the article.  Baccyak4H (Yak!) 15:00, 19 December 2007 (UTC)

text: This would depend on context.  For one observation or data point, no, since X is a row vector.  For the entire data, yes.  I am not sure what you mean by "vector context", but I hope those two scenarios clarify things.  Baccyak4H (Yak!) 16:26, 21 December 2007 (UTC)
thread_title:  Boldface vectors 

message: Several types of regression which fall under this topic have been added to the see also section.  However, links to these already appear where they are discussed higher up in the article. I propose to remove them in the see also section.   Anyone with me here? Baccyak4H (Yak!) 19:12, 2 February 2008 (UTC)
thread_title:  See also 

message: I think the technical term for the family of distributions is the Exponential Dispersion Family, though unfortunately I have no sources to confirm it other than my hazy memory. Can anyone confirm this?
Technically one doesn't even need to specify a distribution to fit a GLM, only a variance function is required (though specifying a distribution means one can estimate the dispersion parameter by maximum likelihood. However I don't know if that can be worked into the article without making it more confusing.
There is no mention in the article of iteratively re-weighted least squares (IRLS or IWLS depending on who you talk to), the method used for estimating the parameters, and the current article in that location doesn't seem relevant GLMs. 
Thoughts? -3mta3 (talk) 00:17, 8 April 2008 (UTC)
replies: text: 3mta3, good points. One does need to specify an exponential family to fit with ML, but you are right that there are pseudo-ML options as well. I think this could be a new seciton. There also really should be a section on IRLS and Fisher steps, this a big part of what brings the GLM together, they can all be fit with the same general solver. Pdbailey (talk) 02:48, 8 April 2008 (UTC)
thread_title:  A couple of points 

message: Formerly, formula of the density function was
f_Y(y; \theta, \tau) = \exp{\left(\frac{a(y)b(\theta) + c(\theta)}
                                                {h(\tau)} + 
                                           d(y,\tau) \right)}.
With that density function, if a is identity function and b is identity function, the mean of the distribution is
\mu = \operatorname{E}(Y) = -c'(\theta)
and the variance is
\operatorname{Var}(Y) = -c(\theta) h(\tau).
Now, in the formula of the density function, the sign in front of c is minus. The formula is
f_Y(y; \theta, \tau) = \exp{\left(\frac{a(y)b(\theta) - c(\theta)}
                                                {h(\tau)} + 
                                           d(y,\tau) \right)}.
I am fine with the change, because the formula is more closely resembles the formula that I saw in a book. But, with that change in the formula of the density function, the formulas of mean and variance are now incorrect.
So, to make the formulas of mean and variance correct, there are two alternatives:
* The formula of density function is changed back (the sign in front of c is plus); the formulas of mean and variance are not changed.
* The formula of density function is in current condition (the sign in front of c is minus); the formulas of mean and variance are changed (the minus sign removed).
--Anreto (talk) 04:57, 8 October 2008 (UTC)
thread_title:  Formula of density function related to mean and variance 

message: This is not as clearly spelled out in the book, but I would encourage you to read section 2.5 and 2.5.1 before you pass judgment on the claim. Now, I can see that the claim might, in order to be completely correct, be taken down a notch. But in precision it loses the ease of reading which is important in a lead. Fisher proposed a method of approximating the Hessian, but this is the key to using Newton's method for this set of problems. Certainly, Nelder and Wedderburn's paper and found that the method could be expanded to the exponential family form. But the Nelder and Wedderburn paper is about fitting the model and less about its application (which the MC&N book covers in more detail) and, again, Fisher made the one of the key insights on this front. Now, if you read this, read sections 2.5 of MC&N, and understand it and still want it out, then by all means, take it out. Alternately, it might make more sense to point it out in the body instead of in the lead, that is fine too. PDBailey (talk) 21:43, 11 January 2009 (UTC)
replies: text: I've just skimmed those sections of McC & N and read the 'Bibliographic notes' in section 2.6 of McC&N, which i think make things fairly clear. As I understand it: Fisher introduced one method ('Fisher scoring') of maximizing the likelihood of the probit model. Wedderburn & Nelder generalized this to all exponential-family distributions and also generalized further by introducing the link function. These two generalizations together constitute the "generalized" bit of generalized linear models, and Fisher had nothing to do with them. I thought your edit implied that Fisher essentially proposed GLMs themselves, while I think he proposed a method (extended by N&W) used in fitting GLMs, which is why I reverted (but i'm happy we're discussing it). I think it would be worth explaining Fisher's contribution in a section on fitting GLMs, but the article doesn't have such a section at present. Qwfp (talk) 23:00, 12 January 2009 (UTC)
replies: text: Agreed, fitting is the right spot for the point to be made. PDBailey (talk) 23:17, 12 January 2009 (UTC)
thread_title:  essentially due to Fisher 

message: About this sentence in the lead:
:"It relates the random distribution of the measured variable of the experiment (the distribution function) to the systematic (non-random) portion of the experiment (the linear predictor) through a function called the link function."
I think that is an accurate description (IIRC I helped write it).  However, it might be considered a little wordy or technical.  I was thinking of rephrasing it to segue with the first sentence, about being a generalization of least squares regression (linear regression).  Something along the lines of, rather than just equating the mean to the linear predictor, it allows for particular relationships between them (in statspeak, the link function is being generalized).  Thoughts? Baccyak4H (Yak!) 17:25, 6 February 2009 (UTC)
thread_title:  Lead sentence 

message: If the reported atomic values were reported and related to the 2 atomic parameters: 1, Deuteron number (=2 times the Z number), and 2, Extra neutron number (= the n - Z number), what would be the best regression method to assign the variance to these 2 factors and determine the best regression equation?WFPM (talk) 03:05, 10 December 2011 (UTC)
thread_title: Atomic mass values

message: An extension is something that expands the generalized linear model: the multinomial models shown are simply special cases of distribution and link.  — Preceding unsigned comment added by 24.34.200.147 (talk) 01:17, 28 May 2012 (UTC)
thread_title:  Multinomial belongs under examples, nout under extensions 

message: The section on HGLMs incorrectly suggested that these models differ from GLMMs.  If what differs is the method for fitting the models, i.e., if there is no difference in the mathematical expression of the relationship between the covariates and the outcomes, then there is no need for a new acronym or to claim a new model has been expressed.  I removed the section.  — Preceding unsigned comment added by 24.34.200.147 (talk) 01:24, 28 May 2012 (UTC)
thread_title:  Conflation of models and methods for fitting models 

message: A line implied that GLMMs "assume" normal random effects.  In fact, there is no such limitation, though abilities to relax this assumption (other than via MCMC) are limited.  The GLMM does not include any such assumption.  Similarly, the GLMM is not (as was implied) limited to single-level models, with multilevel of hierarchical linear models implying a further generalization.  I simplified the "entensions" section to correct this.  — Preceding unsigned comment added by 24.34.200.147 (talk) 01:27, 28 May 2012 (UTC)
thread_title:  Incorrect claims regarding what a GLMM means 

message: Small point perhaps, but I found a bit puzzling on first encounter: Can someone "in the know" add a small note about notation "μ". I find it a bit confusing here, and I found the same some years ago when I first read about GLM from a textbook. Towards the start here it seems to be implied that "μ" will, in the general case, stand for the expected value of the response variable. But then later, this notation is not always adhered to strictly. E.g. it seems to be used that way for Bernoulli response, but not for Binomial or Multinomial (unless you redefine the response to be the proportion, rather than the count, which is not clearly and consistently done).  — Preceding unsigned comment added by 83.217.170.175 (talk) 01:11, 18 September 2012 (UTC)
thread_title:  Notation "&mu;" 

message: Also, it might be a slightly confusing notation because the mean varies with x --- I think it is confusing to leave out x as an argument. Superpronker (talk) 10:26, 19 October 2012 (UTC)
thread_title:  Conditional mean... 

message: The part about binomial regression shows the link function for a Bernoulli variable, without really saying why this works for a binomial.  For instance, if Y is Binom(n,p), then E(Y) = np and g(E(y)) = log((np) / (1-np)) which doesn't make sense.  — Preceding unsigned comment added by Statr (talk • contribs) 19:41, 9 April 2013 (UTC)
thread_title:  Binomial or bernoulli ? 

message: I reworked the complementary log-log section.  A colleague asked me a question that clearly indicated he confused what he read there about the identity link with the cloglog.  I also added a discussion of cloglog and its relation to the Poisson distribution.  I hope most people find this clearer and does not confuse anyone.  DavidMCEddy (talk) 08:34, 19 October 2015 (UTC)
thread_title:  Complementary log-log 

message: This article, after a suitable cleanup, could provide a basis for a section on hypothesis testing in generalized linear models. Though the title and intro suppose binary outcomes, nothing in the discussion requires it. Wikiacc (¶) 04:40, 11 December 2020 (UTC)
replies: text: I actually can't tell if the score test example in that article requires binary outcomes. Regardless, there's usable material there that could be cleaned up and merged. Wikiacc (¶) 04:48, 11 December 2020 (UTC)
replies: text: Closing, on the grounds of not yet; given that no clean-up has taken place over the last year (a requisit for any merge). Klbrain (talk) 12:09, 1 January 2022 (UTC)
thread_title:  Proposed merge of [[Testing in binary response index models]] into [[Generalized linear model]] 

message: In the table "Common distributions with typical uses and canonical link functions", at column "Mean function" and row "Binomial", the formula is wrong. It should be  \mu = \frac{n}{ 1 + exp(-X \beta)} instead of  \mu = \frac{1}{ 1 + exp(-X \beta)}.--148.252.128.186 (talk) 19:20, 15 November 2021 (UTC)
thread_title:  n is missing in the table 

message: when is glm consistent for its estimators? 68.134.243.51 (talk) 14:23, 28 September 2022 (UTC)
thread_title:  no consistency of estimator necessary condition section