Talk:Coefficient of determination

message: The 2021 PeerJ article cited in the introduction has a number of issue to my eye: 
# The main author is an editor of that journal (maybe in good faith, but still a red flag).
# Their definition of R^2 is not the one typically implemented in software and it doesn't match the one in this Wikipedia article. There are 10+ definitions of R^2 and they are not generally equivalent. See Kvålseth (1985) https://www.jstor.org/stable/2683704
# The paper cites forum discussions and blog posts. These may be good quality but it doesn't help raise the bar for the confidence in the material.
# The writing seems sloppy - minor issue, but doesn't help the overall image.
About the statement in the Wiki article: I find "truthful" to be very misleading. Without getting philosophical, the matter at hand is assessing regression. One main point of the paper is that R^2 is more intuitive since it can be seen as a percentage, while the other measures have arbitrary values. I rephrased accordingly, though not sure how to feel about the cited paper. SpunkyLepton (talk) 20:25, 11 November 2021 (UTC)
thread_title:  Problematic article? 

message: It says that only in the case of a linear regression, the coefficient of determination is equal to the square of the correlation coefficient. Should this not be: only in the case of a linear regression with a linear model? (a linear regression can also be performed with e.g. a quadratic model, in which case the the coefficient of determination is not equal to the square of the correlation coefficient). Job. 193.191.138.240 (talk) 09:53, 9 April 2008 (UTC)
replies: text: "With a linear model" would be ambiguous at best considering that in a regression context a "linear model" refers to how the parameters relate to the predicted values, not the structure of the model with respect to changes in the explanatory variables. Anyway, the present version is almost clear and nearly correct ..."correlation coefficient between the original and modelled data values" is meant to mean the correlation between the observed and predicted values, not the correlation between the observed and individual explanatory variables. Unfortunatey previous changes have left some terminology a liitle vague. I will try to improve things. Melcombe (talk) 11:24, 9 April 2008 (UTC)
thread_title:  link with correlation coefficient 

message: This page is confusing because the variables are not consistent with the pages for Residual sum of squares and Explained sum of squares. On those pages, SS_{E} is defined as ESS and the same for SS_{T}, SS_{R}. Also, isn't RSS=\sum{(y_{i}-\hat{y}_{i})^2}? Don't want to fix this myself because I'm just learning stats, maybe someone more experienced can?  —Preceding unsigned comment added by 128.6.30.208 (talk) 03:22, 23 October 2007 (UTC)
replies: text: The formuls for R2 involving SS_{reg} should only be used in those cases where \bar{f}=\bar{y} so there would be no difference. Melcombe (talk) 15:30, 11 September 2008 (UTC)
thread_title: Definitions/Variables are not consistent here

message: There is a bit better explanation of this at:
http://www.csus.edu/indiv/j/jensena/mgmt105/adjustr2.htm
I think we can add to the definition:
1) the motivation for "Adjusted R Square". And 
2) to note that it can be viewed as an Index when comparing regression models (like the standard Error).
Tal.
—The preceding unsigned comment was added by Talgalili (talk • contribs) 16:04, 21 February 2007 (UTC)
thread_title: Adjusted R Square

message: Who says R-squared should be greater than zero? For example if measured y-values are between 9 and 10, and model prediction is always zero, then R-squared is heavily negative.Kokkokanta 07:50, 28 January 2007 (UTC)
thread_title: Range of R-squared

message: * Consider mention of the Nagelkerke criterion, an analogue 
that you can use with generalized linear models, which 
are not fitted by ordinary least squares.
* We can't assume that R^2 is applicable with every kind of 
least-squares regression.  For example, it doesn't make sense 
with regression through the origen.  There has been a discussion of limitiations, 
in American Statistician. 
* Adjusted R^2 can be negative.
Dfarrar 14:04, 8 March 2007 (UTC)
thread_title:  Possible expansions 

message: R^2 is only one measure of association.  The causality issue applies to all of them.
The issue has been addressed generically.  See links inserted.
Dfarrar 14:29, 8 March 2007 (UTC)
thread_title:  Causality 

message: This has been a good day for additions to my watched pages.
Regarding this new material, I think some terms could be explained 
to make the article more widely accessible, without doing much
harm, e.g. "weakly smaller."  Repeating a previoius point, I suggest
inclussion of material on analogous statistics applicable with models
other than Gaussian, e.g., with generalized linear models.
Dfarrar 22:25, 20 March 2007 (UTC)
thread_title:  Inflation of R-square 

message: I changed the formula to what I believe to be the correct one, but it has been reverted. My source is Essentials of Econometrics by Damodar Gujarati. Can whoever changed it please cite their source for this? Cheers.
* I can see why you're confused.  In your book though, "E" likely stands for "Explained" and "R" likely stands for "Residuals."  In the equation on this page, "R" stands for "Regression" (or "Explained") and "E" stands for "Error" (or "Residuals").  Gujarati's Basic Econometrics uses "Explained" and "Residuals" as well, so the lettering is exactly the opposite.  VivekVish 03:58, 18 April 2007 (UTC)
thread_title:  R squared formula 

message: This bit seems wrong to me: "adjusted R2 will be more useful only if the R2 is calculated based on a sample, not the entire population. For example, if our unit of analysis is a state, and we have data for all counties, then adjusted R2 will not yield any more useful information than R2."
It is not clear why this would be. Even if you had the population, you would still be concerned about exhausting degrees of freedom. You would thus want to penalize any calculation of the R2 for the number of regressors. If you have the population of U.S. states (N=50) and you have a model with k=50, you will perfectly predict and get an R2 of one. But this is misleading. The adjustment is meant to account for degrees of freedom, not estimation error. 
Still A Student 03:06, 9 September 2007 (UTC)
thread_title:  Adj R2 

message: A simple question but I cant figure it out:
Why is r square for y=(1 3 5) and y_est=(2 7 3) is bigger
than 1? It must be between 0 and 1.
SSR=17 SST=8  —Preceding unsigned comment added by 85.107.12.120 (talk) 13:21, 20 September 2007 (UTC)
replies: text: Attempted improvement. Melcombe (talk) 10:06, 6 November 2008 (UTC)
thread_title:  R-squared bigger than 1? 

message: What is \bar{y}? is it the mean? Can someone put that in the text?
--Play
have included a first version Melcombe (talk) 12:12, 6 December 2007 (UTC)
thread_title:  What is <math>\bar{y}</math>? 

message: I believe that R-squared is a measure of variability aligned rather than variability accounted for. With respect to correlation is not causation, consider R-squared as variabilty "aligned" rather than "accounted for." For example, if the number of churches in cities is correlated with the number of bars in cites, say .9 ,  then R-squared is .81. Rather than number of bars accounting for number of churches, consider that variability (81%) of their related numbers is aligned. (Their variability alignment is most likely "accounted for" by population.) Respectfully submitted, Gary Greer greerg@uhd.edu  January 26, 2008.  —Preceding unsigned comment added by 75.16.159.122 (talk) 01:34, 27 January 2008 (UTC)
replies: text: Seems to me this dicussion points to a shortcoming in the article. It uses terms such as "explain" and "account for" without prominently mentioning that these are terms of jargon. Civilians often find news stories saying that "statistics show" that a particular variable, such as personal income or running speed, is explained by 10% or 45% or whatever, by race or height or some such, and assume that the explanation for the other 90% or 55, or whatever, might also be found. Somewhere near the top, the article should warn that these various "accounts for" numbers imply neither causality nor that they ought to up to 100%. Jim.henderson (talk) 15:50, 1 May 2012 (UTC)
thread_title:  Causality Again (was at top) 

message: I undid a set of changes that tried to change the notation from  R2 to r2 ... because -
*  I think R2 is the most commonly used notation
*  The notation was not changed everywhere, specifically in displyed maths and section titles and possibly elsewhere, so that the result as left was very poor. 
Melcombe (talk) 10:09, 6 March 2008 (UTC)
thread_title:  Undid change from R<sup>2</sup> to r<sup>2</sup> 

message: Forcing the linear curve through the origin is the same as adding the point (0,0) to your data and giving it an infinite number of replicates. If we believe that (0,0) is a legitimate and absolute point, then by using this point to fit our curve, we can argue that the accuracy of our curve is improved merely because we know that it goes through at least one point that we believe to be correct. By weighting the origin this way, we could argue that the coefficient of determination should use the following equation, when it is forced through the origin.
R² = (SUM xy)² / (SUM(x²) * SUM(y²))  
A derivation of this formula can be provided if you want to see one.
—Preceding unsigned comment added by JNLII (talk • contribs) 22:36, 11 November 2008 (UTC)
thread_title:  Weighting the origin as an absolute known point 

message: Per Wikipedia's guidelines, at least the summary of this article should give nearly any person a good understanding of what this is. As it stands now, the summary only states "this isn't defined but were going to give you a 3 page admittedly non-defining definition which only people who have studied statistics will understand".
(I'd like to second this point; I came here to find computing formulas for the coefficient of determination for polynomial regression, and found three pages of incomprehensible and useless (to me) blather.)  — Preceding unsigned comment added by Alan8 (talk • contribs) 02:02, 10 June 2014 (UTC)
thread_title:  Article needs to have summary which is understandable to common people 

message: Is
\bar{f} = \frac{\sum f_i}{n} \mbox{ with n the number of data points}
or is
\bar{f} = \frac{\int f(x) dx}{\int dx} 
?
Often, f is a continuous model and a mean can be defined according to the second equation. This seems to make more sense. However, the text seems to imply that the first formular needs to be considered, even if f(x) is a continuous function.
I hope, the question became clear. 
Tomeasy T C 08:55, 29 June 2009 (UTC)
replies: text: I have read the relevant parts of the article several times and cannot understand the point being made in the above comment. However, it may or may not be relevant to point out that a sample is always finite and discrete, no matter what the nature of the population. JamesBWatson (talk) 20:09, 29 September 2009 (UTC)
replies: text: Note that f does not refer to the sample but to the model (or regression) fitted to the sample. Tomeasy T C 18:28, 1 October 2009 (UTC)

text: I have now thought about this again. I guess that the question is probably intended to mean "is  \bar{f}  the mean from the sample or from the population?" If so then the answer is that  \bar{f}  is the mean of the modelled values using the model parameters calculated from the sample. JamesBWatson (talk) 09:57, 1 October 2009 (UTC)
replies: text: Again, f is neither the sample nor the population, f is the model. So, this was still not what I intended to ask. As the model, f can be a continuous function whose mean can be evaluated according to the second equation above. Of course, one could evaluate f at the discrete points where data exists and compute the mean according to the first equation. Since the results are not necessarily equal, I am asking what is meant. Tomeasy T C 18:28, 1 October 2009 (UTC)
replies: text: I think you should always be using the definition given immediately after "The most general definition of the coefficient of determination is", which does not involve  \bar{f} . It would be possible to replace the sums by integrals there, but the key is that the comparison of y with f can only take place for points for which the observed values are available. So you could use integrals only if y is observed continuously, assuming that if f is computed discretely you would include a rule for interplating, so producing a continuous version of f. Melcombe (talk) 09:10, 2 October 2009 (UTC)

text: I have continued to wrack my brains in a further attempt to understand this question, and I think I have got it: at any rate I hope so. I suppose the intention is that "model" is intended to mean the  function estimated from the sample. In this case it is true that f is commonly a function on an interval, rather than on a discrete set of points. Nevertheless, the sum of squares SS_{\rm tot}=\sum_i (y_i-\bar{y})^2 is calculated from values at a discrete set of values of x, and it is this discrete sum which is partitioned into "regression" and "residual" components. Therefore all the sums involved take place on the same discrete set of values of x. It is also worth mentioning that f is not necessarily a function on an interval: it may be only a discrete set of values. JamesBWatson (talk) 19:15, 4 October 2009 (UTC)

text: Usually (as explained further down the article) the two are the same, so it makes no difference. However, if they are not the same then it seems to me that it should be \bar{y} so that the "total" sum of squares really is the total of the other two. JamesBWatson (talk) 11:01, 14 April 2010 (UTC)
replies: text: That's actually another thing I would contest. Why should the following hold: SSE_tot = SSE_reg + SSE_err. Obviously, it would be true if the terms were not squared - but they are. Even using the knowledge about y_bar being the mean of all y_i does not make the addition of two sums equal to SSE_tot. It may become true, if you apply special constrains on the model values f_i. However, the notations we discuss are given within a general context. Tomeasy T C 18:45, 14 April 2010 (UTC)
thread_title:  f^bar is mean of values or function? 

message: I think the distribution of R2 under the null hypothesis of independent, normal dependent and independent variables should be included. it's some kind of transformed F statistic, I believe. Shabbychef (talk) 17:00, 11 September 2009 (UTC)
thread_title: distribution of R<sup>2</sup> under the null?

message: I removed the addition of "A practical minimum for R2 is 0.80 when the correlation is used for predicton." because
:*it is not appropriate in the lead;
:*it is not generally true and any criterion of usefulness will obviously depend on the actual context;
:*there was no citation.
Melcombe (talk) 10:53, 28 June 2010 (UTC)
replies: text: Good call. Talgalili (talk) 07:27, 29 June 2010 (UTC)
thread_title:  Deletion of addition 

message: I believe the literature standard is to use "errors" to refer to the errors of the true model (with true but unknown coefficients) and "residuals" to refer to the regression residuals.  As far as I can see this article follows that standard in the use of words but not the use of subscripts: specifically, SSerr is always used for the sum of squared residuals, leading for example to this odd definitional passage in the "Definitions" section":  SSerr=... the sum of squared residuals".  And in the adjusted R2 section, VARerr is used for the variance of the residuals.  This notation is likely to confuse the reader as to the distinction between errors and residuals and as to when one or the other is being referred to.
Would anyone object if I go through and change all the "err" subscripts to "res" or "resid"?  Also, how about if I change SSreg to SSregr everywhere, since "reg" brings to my mind "regular" instead of "regression"?  
Also, in the Regression analysis article, I want to change "SSE", defined there as the sum of squared residuals, to "SSR".  Comments?Duoduoduo (talk) 18:27, 19 November 2010 (UTC)
thread_title: Terminology: errors vs. residuals

message: This article talks about adjusting for the number of degrees of freedom in a fit with a certain number of variables like regression coefficients.  How does one adjust for trying multiple models, each with potentially different numbers of variables, such as in symbolic regression? Ginger Conspiracy (talk) 04:21, 24 December 2010 (UTC)
replies: text: One would start with the most obvious thing.... which I think in this context would be to set up a simulation experiment that replicates all the different steps in a "typical analysis" of the kind concerned, including any use of multiple models with different numbers of parameters. There is no real point in looking for an "adjusted number of degrees of freedom" since that is unimportant and mostly a fictional concept. Rather one would concentrate on a measure of fit, such as R2 (or it may easier to use just a sum of squares of errors), and look at the disrtibution of this test statistic (after all the model selection steps), as derived from multiple simulations. Melcombe (talk) 09:32, 24 December 2010 (UTC)
thread_title:  Adjustment for symbolic regression 

message: 99th percentile in math, but rusty and in the limited time available came away with but little grasp of what the coefficient of determination means or how it's used.  
Those of us who are visual more than mathematical can get the story at a glance if shown two charts, one with high correlation coefficient, one low.  
Even the math-able will be introduced to the concept more quickly and easily if the charts are provided in examples at the beginning as an introductory over "view."  — Preceding unsigned comment added by 66.167.61.181 (talk) 14:40, 4 September 2011 (UTC)
thread_title:  Some high- and lo-correlation charts would help readers grasp the concept at a glance 

message: Here's hoping for an article more useful to all.  — Preceding unsigned comment added by 66.167.61.181 (talk) 14:42, 4 September 2011 (UTC)
thread_title:  It would also help to have several examples of how R2 is used. 

message: Maybe it is already present elsewhere on Wikipedia, but I couldn't find the phrase "norm of residuals" on Wikipedia in a Google search. I think it would be helpful to have a clear equation for norm of residuals (as is commonly used in Matlab).  Maybe this would just be for me mostly, but just seeing the equation \text{norm of residuals} = \sqrt{SS_\text{res}}   (please correct if this is wrong) or something similar would help I think.  I like the thought of putting that on this page because of the visuals and equations already present.  The page for explained sum of squares could also be a candidate (though it has no figures).  Is there a good place to put the norm of residuals equation?
Jacobkhed (talk) 23:57, 8 July 2013 (UTC)
replies: text: Sorry for late response, User:Jacobkhed. MATLAB's resnorm (norm of residuals) is same as RSS. Therefore, that statement is incorrect, and also the cited source doesn't mention MATLAB. EyeTruth (talk) 06:08, 25 February 2016 (UTC)
replies: text:  Indeed, matlab's documentation is inconsistent: first it says "Norm of the residual" then "squared 2-norm of the residual"; only the latter is correct. I've notified support@mathworks.com fgnievinski (talk) 20:53, 25 February 2016 (UTC)
replies: text:  Norm of Residuals is false. residual Standard error would be correct.--JonskiC (talk) 12:06, 20 January 2018 (UTC)
thread_title:  Norm of Residuals? 

message: Given that the R2 value is a comparison with the null hypothesis that the sample set is best represented by the constant function at the value equal to the arithmetic mean of the sample set, it's really an important caveat that the power of the R2 value to explain variability is directly slope-dependent.  In other words, if a sample set function is constant or nearly constant, as opposed to strongly increasing or decreasing, the R2 value for the data set will be low, because the test cannot distinguish the trend (stable) from the null hypothesis (a constant of the arithmetic mean).  I'm not a statistician, so certainly someone else might be better off writing up something to describe this caveat to a lay audience, but what I'm describing is important information that should be presented plainly and not in mathematician-ese.  — Preceding unsigned comment added by 12.130.161.8 (talk) 01:24, 8 February 2014 (UTC)
thread_title:  Slope-Dependency 

message: There is twofold common use of notation p in multiple regression:
(i) $p$ as the number of predictors (as in here), 
and (ii) $p$ as the number of parameters in the linear predictor. 
The attribute LINEAR in linear regression 
concerns not X's but betas. Thus the dimension of the model
is the dimension of beta space (second dimension of the design matrix).  
Many authoritative books in the field  use $k$ to denote 
the number of  predictors $(X_1, ..., X_k), $ and $p=k+1$ as the number of parameters 
$(\beta_0, \beta_1,..., \beta_k).$
Of course this is only a convention, but confusions are possible
(if you check  for the definition of Mallows' $C_p$ there is a mess on the web
with $p$ standing for both).
Maybe a sentence pointing out this dichotomy will be prudent.
Greetings from Marietta GA  — Preceding unsigned comment added by 71.204.20.163 (talk) 16:28, 15 February 2014 (UTC)
thread_title:  p -- number of variables or number of parameters 

message: I'm wondering about this one:
 "Seventy percent of the variation in the response variable can be explained by the explanatory variables. The remaining thirty percent can be attributed to unknown, lurking variables or inherent variability."
Shouldn't it be
 "Seventy percent of the variance in the response variable can be explained by the explanatory variables. The remaining thirty percent can be attributed to unknown, lurking variables or inherent variability."?
I guess variance is a measure of variation but when I read "variation" I would not expect variance...
Bgst (talk) 10:43, 11 September 2014 (UTC)
replies: text: Go for it.  Dger (talk) 00:29, 18 September 2014 (UTC)
replies: text: Done! Bgst (talk) 07:57, 19 September 2014 (UTC)
thread_title:  Interpretation 

message: Consider the observed data set {10,10,10}. For this data set we see that:
: i=3
: y_1=10
: y_2=10
: y_3=10
: \bar{y}=10
: y_i=\bar{y} for all i
And the total sum of squares is zero, i.e.
: SS_\text{tot}=\sum_i (y_i-\bar{y})^2=(10-10)^2+(10-10)^2+(10-10)^2=0
And since the formula is
: R^2=1-SS_\text{res}/SS_\text{tot}
That means we have a division by zero thus:
: R^2=1-SS_\text{res}/0
...and the formula is not defined. Regards,Anameofmyveryown (talk) 11:18, 22 February 2015 (UTC)
thread_title:  Worked example when yi = ybar for all i and i>1 

message: Consider the observed data set {10}. For this data set we see that:
: i=1
: y_1=10
: \bar{y}=10
: y_i=\bar{y} for all i
And the total sum of squares is zero, i.e.
: SS_\text{tot}=\sum_i (y_i-\bar{y})^2=(10-10)^2=0
And since the formula is
: R^2=1-SS_\text{res}/SS_\text{tot}
That means we have a division by zero thus:
: R^2=1-SS_\text{res}/0
...and the formula is not defined. Regards,Anameofmyveryown (talk) 11:18, 22 February 2015 (UTC)
thread_title:  Worked example when i is equal to 1 

message: This is a bit outside my comfort area, but I have lately been evaluating linear mixed models using a formulation for deriving a marginal R2 (that is, based on fixed effect(s) only) or a conditional R2 (that is, conditional on all fixed and random effects in the model). This is based on two papers by Nakagawa et al. that frankly have gotten the shit cited out of them since their publication. In practical application, this was apparently very well received because it allows estimating model fit while still honoring the random structure of the data. However, it doesn't look as if this approach is covered here yet. A sentence or two should fit well under Coefficient_of_determination#In_a_multivariate_linear_model. There's an R implementation in package MuMIn (::r.squaredGLMM()) that might be worth mentioning - for those of us who scan these articles in the hopes of tackling a practical problem.
I could whip up a bit of text but would have to keep it very general because maths be hard. So if someone else is interested in doing that, it might be preferable? --Elmidae (talk · contribs) 22:21, 7 October 2020 (UTC)
thread_title:  Add discussion of marginal and conditional R<sup>2</sup> for multivariate models? 

message: Thanks for adding a mention of the Olkin-Pratt estimator.  
What do you think about adding a bit more about this?  For example,  expression (2.3) in the Olkin-Pratt paper you cited gives a simple expression in terms of the well-known Gaussian hypergeometric function.  And the MBESS::Expected.R2 contributed function for R (programming language) has it programmed, so anyone who wants it can easily add it to any relevant analysis in R.  DavidMCEddy (talk) 16:49, 23 April 2021 (UTC)
replies: text: Hello, it was not me that did the addition, all I was doing was correcting a cite date error. I have no knowledge of the topic. Sorry. Keith D (talk) 16:54, 23 April 2021 (UTC)
thread_title:  Olkin-Pratt 

message: A further interpretation of R^2 was added by an anonymous editor who cited a recently self-published note on internet [I'm not saying that the editor and the author of the note are the same person. Just pointing out the note was not formally published in the academic literature]. The text referred to “quotients of surfaces” which makes no sense. Richard Gill (talk) 13:07, 11 February 2023 (UTC)
thread_title:  New passage removed