Talk:Multinomial logistic regression

message: This page appears to fail to recognize the existance of ordered multinomial responses, no? Pdbailey 02:15, 12 June 2007 (UTC)
thread_title: 2007

message: In the Model, we better have the probabilities adding to 1:
:\sum_{j=0}^{J} Pr(y_i=j) = 1
That means the denominator needs to be:
:1+\sum_{j}^{J}\exp(X_{i}\beta_{j})
and not just
:\sum_{j}^{J}\exp(X_{i}\beta_{j})
The 1 here is the contribution from Pr(y_i=0).
By the way, I'm used to thinking of X_i as a vector, in which case X is a matrix, \beta_j is a vector, and X_i \beta_j is a dot-product.  The way these equations are written imply that X_i and \beta_j are scalars, which is okay if you are predicting y from a single scalar X.  I found that a bit confusing and had to stare at it a bit longer than usual to parse it.  Some clarification about this might be helpful in the article.
I believe the summation in the denominator is over an incorrect index. If there are K types of probabilistic events and J explanatory variables x1...xJ, the summation in the denominator should be over k to K k=2,3...K.  The missing k is the base probabilistic event.  The implicit summation embedded in the vector multiplication XB is over j from j=1 to j=J.  —Preceding unsigned comment added by 72.13.225.74 (talk) 17:13, 20 October 2008 (UTC)
replies: text: The '1's most definitely need to be there, so I've reverted the change that removed them. They can be derived by assuming \log \frac{Pr(y_i = j)}{Pr(y_i = 0)} = x_i^\top\beta_j, j=1, \ldots, J. 209.2.237.95 (talk) 01:01, 2 March 2011 (UTC)
thread_title:  I corrected a mistake in the denominator 

message: I changed "If the multinomial logit is used to model choices and the error terms are assumed to be independent, it can violate the assumption of independence of irrelevant alternatives (IIA)."
The IIA stems from the independence of the error term. The IIA is violated not by the independence of the error term, but by the non independence in practice. In any case, as there is no formulation of the MNL with an equation with errors and their distribution, I do not think we should describe the IIA in terms of structure of errors as this will not make the article clearer. The bus example is better. Also the nested logit does not allow for correlated error but for different variances for the error term in different subgroups within which the IIA still holds (see for instance Green Econometric Analysis, pp. 847-850). Gpeilon (talk) 20:22, 3 November 2009 (UTC)
replies: text: I am not arguing against removing the bus example, I think we should keep it. I think we disagree about IIA however. I think it is a desirable property that a model can have for discrete choice (see for example the bus example), and that an iid multinomial logit does not have this property. However, allowing for correlated error terms, IIA can be satisfied. Do you agree with that? PDBailey (talk) 20:36, 3 November 2009 (UTC)

text: Hmm, I think there is some confusion here. The Wikipedia article on IIA states a definition that is at odds with i.e. this article. In the bus example, the ratios or red bus to car change in fact, but in the model they do not. This means that in the model the IIA assumption is violated (some people move from car to bus despite having picked car before the blue bus was added), but the model is supposed to have the IIA property according to the linked preference. PDBailey (talk) 20:42, 3 November 2009 (UTC)
thread_title:  IIA 

message: As a student i think a visual representation might be nice, i would make one my self but i am just starting to learn R.  —Preceding unsigned comment added by Thedreamshaper (talk • contribs) 21:51, 28 October 2010 (UTC)
thread_title:  Visual request 

message: An IP recently deleted the "assumptions" and "estimation of the intercept" sections. I strongly disagree that this makes the article better, so I undid this. Any objections? 018 (talk) 14:45, 27 December 2011 (UTC)
thread_title:  deletion of assumptions/estimation of the intercept sections 

message: Some equations in the section Multinomial logistic regression#As a set of independent binary regressions have some parameters \beta' which as far as I can see are not defined, and it appears to me that they ought to be just \beta. Any objection to my removing the prime signs? Duoduoduo (talk) 17:54, 5 February 2013 (UTC)
thread_title:  Unexplained notation 

message: Perhaps I'm missing something obvious, but the section "As a set of independent binary regressions" starts by talking about doing K-1 regressions and then presents formulas for the log probability ratios such as Yi=1 / Yi=K.  If we were to run this as, say, a logistic regression, what would the dependent variable be?  1 if the outcome is 1 and 0 if the outcome is K, but what about Yi=2 to K-1?  — Preceding unsigned comment added by Jetopal (talk • contribs) 01:10, 26 January 2014 (UTC)
thread_title:  Clarifying the section "As a set of independent binary regressions" 

message: The stated function is not actually the softmax – it's missing x_i in the numerator – and so it approximates the indicator function 1(x_j = max_i x_i) rather than max_i x_i, as is stated in the article. — Preceding unsigned comment added by 162.129.251.86 (talk • contribs) 
:The function is correct, but the explanation is not. The softmax is supposed to approximate the argmax, not the max. I'll fix it. QVVERTYVS (hm?) 14:36, 7 October 2014 (UTC)
replies: text: I stumble on that paragraph as it stands now. While the softmax function approximates the indicator function, it is/was indeed correct to state that it can be used to construct a weighted average that approximates the max function: The dot product of the vector x and softmax(x). Elias (talk) 19:35, 8 March 2023 (UTC)
replies: text: PS: While x dot softmax(x) is a smooth approximation of the max function, the LogSumExp function apparently is more common for this purpose. softmax is the gradient of the LogSumExp function. I suggest removing some of the low-quality text about softmax from this article and refer to Softmax instead.  — Preceding unsigned comment added by Ehasl (talk • contribs) 10:11, 10 March 2023 (UTC)

text: PPS: Apparently, the approximation I describe has been studied. It is named the "Boltzmann operator", and includes a parameter α that uniformly scales the softmax inputs. With α→∞, the function converges to the maximum, and with α→-∞, to the minimum: x⋅softmax(αx)  — Preceding unsigned comment added by Ehasl (talk • contribs) 10:35, 10 March 2023 (UTC)
thread_title:  Statement about softmax is incorrect 

message: I saw you just added this page to :Category:Categorical data, but I'm not sure if that's appropriate. The only thing necessarily categorical in LR are the class labels (the outcomes/dependent variables), but that is true for any classification model. The inputs (independent variables/predictors/features) are real values. Perhaps :Category:Classification algorithms should be a subcat of :Category:Categorical data? QVVERTYVS (hm?) 18:00, 26 May 2015 (UTC)
replies: text: Okay, I'd have no objection to your doing that. I see that logistic regression is also under Category Categorical data (that's what led me to add the category here), so if you subcategorize it like that maybe you should also move logistic regression to category classification algorithms (and put probit there too). Loraof (talk) 19:57, 26 May 2015 (UTC)
replies: text: Ok, did it! QVVERTYVS (hm?) 19:00, 27 May 2015 (UTC)
thread_title:  Categorical data? 

message: Why is there no worked example?
Mark W. Miller (talk) 11:36, 28 November 2015 (UTC)
replies: text: That level of detail would be more useful for a standard toy problem, to get past introductory material. Let's agree on a standard problem, first. --Ancheta Wis    (talk  | contribs) 14:43, 28 November 2015 (UTC)
thread_title:  Why is there no worked example? 

message: In the introduction, it says that multinomial logistic regression is a solution to a classification problem. This is correct, but not complete. As a regression method, it is also used to find out how the independent variables are related to the dependent variable, in this case by getting odds ratios.  Sometimes classification is not a goal at all. 
Where should that go?  — Preceding unsigned comment added by PeterLFlomPhD (talk • contribs) 12:51, 24 August 2017 (UTC)
thread_title:  Not just for classification Comment 

message: This page is in desperate need of citations. There are a few in the introduction, but most sections present the material with no references at all.66.71.17.135 (talk) 17:18, 7 September 2018 (UTC)
thread_title:  Citations