Talk:Effect size

message: Grant:
I've been reading some more about meta-analysis and effect sizes in relation to the Wikipedia article.
The article now contains two ways of computing an effect size, but it seems that neither is the one that is most commonly used, which is the difference in means divided by the pooled standard deviation. You refer to the denominator provided in Cohen's d as the pooled standard deviation, but I see this also referred to as the mean SD,  reserving the term "pooled" for when the two sample sizes are taken into account.
Although I haven't seen Cohen's original formulation for effect size, from what I have read it seems he specified only that it is the difference in means divided by a standard deviation and that the standard deviation of the control group was originally most commonly used. Then other standard deviations were used (e.g., mean and pooled) and other adjustments made (as in Hedges' g).
So what appears to me to be the most commonly used index of effect size, that is the difference in means divided by the pooled SD (taking the sample size of each group into consideration) is not given in the Wikipedia article. This is also the effect size obtained if it is generated from a t value ( ES = t (sqrt ((n1 + n2)/n1n2))), F ratio, or exact probability for a t-value or F-ratio (see Lipsey and Wilson, 2001, pp. 173-175) --reference added to Wikipedia article.
So I suggest that the article be organized a such:
# correlation coefficient and r-square
# Cohen's d as difference between two means divided by a standard deviation, originally SD of control group
#. . . . divided by mean of the two SDs (as it currently appears).
#. . . . divided by the pooled SD, and how this value can also be computed from a t, F, or exactly provability of a t or F
#. Hedges' g with an explanation of what the second part of the formula does.
#. Odds ratio
This would show the "evolution of the effect size and the basic different ways to calculate it.
I could take a whack at this myself, but still have to figure out how to write formulas on the Wikipedia and I probably won't have time to do this for a while.
Let me know what you think. I also added this to the article's discussion so this doesn't become just a private discussion between you and me.
--Gary Gary 11:52, 4 April 2006 (UTC)
thread_title:  Evolution of effect size 

message: 1. Add discussion of the f effect size measure for ANOVA F-tests.
2. Add discussion of the w effect size measure for Chi-Square tests.
--DanSoper 07:56, 11 June 2006 (UTC)
thread_title: ToDo list

message: "Pearson's r correlation is one of the most widely used effect sizes. It can be used when the data are continuous or binary".
This provoked a discussion as to whether this should say "continuous or discrete".  I wonder if this could be clarified.  I would assume that discrete numeric data were also fine (though not discrete qualitatitive "levels").  --Richard Clegg 16:36, 10 August 2006 (UTC)
thread_title:  continuous or binary 

message: For those of you who don't know, Grant has requested mediation on this issue, and I've taken up the case. Apologies for getting to it a little later than intended.
Now, looking at the Summary section of the article, I see that there are two other examples given. I'm willing to bet a week's mediator wages that something very similar to each can be found in a textbook somewhere. And yet, I don't think that they are in need of citations. Why not? I'll pull out the part of WP:CITE I think is most relevant:
:When writing a new article or adding references to an existing article that has none, follow the established practice for the appropriate profession or discipline.
The important question is, then, under what circumstances an example would be cite-worthy in a scientific paper or text. I think that would be the case only if the example is generally associated with at least one person or work. That the same idea is expressed elsewhere is not in itself sufficient.
Grant has provided some sources which support his claim that the aliens example is somewhat ubiquitous; on the other hand, even the original objection to the example didn't actually give any detailed reason to suppose otherwise. Given that, I'm not seeing the case for citation here.
On the basis that I need food, I'm going to stop here and invite comment from both parties, along with anyone else who's interested. Tsumetai 18:50, 18 November 2006 (UTC)
replies: text: Thank you Tsumetai for taking up the case. I suspect that the 'Aliens' example is now just going to upset some people whenever they see it. Therefore, last night, I changed the example to 'imagine visiting england without any previous knowledge of humans...' (in the summary). I then referenced the data which I used in the example (Health Survey for England, 2004) and continued with that example to demonstrate Hedges' g as well. I hope that this satisfies everyone - I can't see an angle, at the moment, that would make this example untenable. Thanks again for taking the time to look at this Tsumetai. I think Chris might be OK with the changes I made. Best, Grant 02:02, 19 November 2006 (UTC)
replies: text:  Chris obviously doesn't have any objections to this version so I'll keep it as it currently stands. Thank you Tsumetai for taking the time and looking at this issue. Best, Grant 07:48, 30 November 2006 (UTC)
thread_title:  Alien Example: Mediation 

message: Hi Chris & Mycatharsis, Before I revert anything I thought I should just put a discussion comment first. I see what Chris is saying, but must agree with Mycatharsis. Cohen proposed the interpretations suggested in both the 1988 and 1992 papers. The argument he uses is that r is the most fundamental effect size measure available - allowing for direct interpretation of the degree of association. Any problems form anyone if this is reverted and expanded upon? Cheers, Grant 10:56, 29 December 2006 (UTC)
replies: text: What are you referring to? Please place this conversation wherever the rest of it is because it doesn't make sense. — Chris53516 (Talk) 15:10, 29 December 2006 (UTC)
replies: text:  Hi Chris, sorry this didn't make sense to you - I just ran with the last edit made and wrongly assumed everyone was in my head space. I was referring to the edit changes you made here  and was discussed on Mycartharsis' talk page here . Is this the info you needed? Cheers, Grant 13:10, 30 December 2006 (UTC)

text:  I'm afraid I must disagree here. Indeed, r-squared is commonly used because it gives a direct amount of variance that is attributable to the relationship (even if there is a third factor influencing the relationship). r is frequently used as the 'base' effect size and many meta-analyses convert to r rather than d or g (r-squared is exactly that - r multiplied by r). It is frequently referred to as the effect size correlation. A good resource (given on the article page too) is Becker's pages who gives the various formulae to convert to r . We must also keep in mind that effect sizes are directional only by virtue of their + or - sign: the same as r (all depends upon how you set this up in the analyses and whether + is a good or bad thing). I take your point with the ice-cream example, but all effect sizes are subject to this (the notorious Type III problem) and so effect sizes never establish causation, but interpretation may imply it. Cohen gave the rules of thumb (small = 0.1; medium = 0.3; large = 0.5) in his 1988 treatise and 1992 Psych Bull paper. A better explanation is provided by: Rosnow, R. L., & Rosenthal, R. (1996). Computing contrasts, effect sizes, and counternulls on other people's published data: General procedures for research consumers. Pyschological Methods, 1, 331-340. This has always been my interpretation. ps. Happy New Year from down-under everyone. Grant 13:45, 2 January 2007 (UTC)
replies: text: I'm not really satisfied with that explanation. r and r-squared are not measures of effect size; merely of the goodness of fit of the model. R squared likewise tells you nothing about the effect SIZE, only the percentage of the dependent variable which is being affected. The Beta coefficient indicates effect size; it indicates the direction and magnitude of the relationship between the variables. You need to actually think about what the measures are telling us. r and r squared tell us what portion of the variance in the dependent variable is varying in common with the independent variable - correlation - but tell us nothing about effect size, which is the coefficient of the equation of the line. That is, how much change in Y is caused by a one unit change in X. THAT is effect size as it shows you what one variable is doing to the other. r and its derived cousins are measures of the goodness of fit of the model, but tell you nothing about the effect size, only how accurate your model is. This should be immediately obvious; it is perfectly reasonable to find a very high correlation with a very low effect size. When you're talking about r you are only talking about the effect itself, not the magnitude of the effect. 59.167.60.92 (talk) 11:54, 17 January 2009 (UTC)

text: Yes, where are these ranges coming from for r. I don't have the Cohen (1988) in front of me, but I don't see these ranges in Cohen (1992).  — Preceding unsigned comment added by Schmitta1573 (talk • contribs) 14:50, 25 September 2011 (UTC)

text: It's just a term, but I disagree that r-squared is not an effect size.  Following the argument of 59.167.60.92, only a regression coefficient or mean difference (something on the scale of the data) would be an effect size.  Even Cohen's d would be excluded.  But the important distinction is not relative (r-squared, Cohen's d) versus absolute (regression coefficient, mean difference) measures, it is whether the statistic is an inferential quantity like a Z-score or p-value that scales with sample size (definitely not an effect size), or something like the correlation coefficient, regression coefficient, odds ratio or Cohen's d that estimates a relative or absolute population parameter that is independent of the sample size.  In the case of the correlation coefficient, it is unfortunately common to read that a sample correlation coefficient is significantly different from zero, perhaps with some p-value, but then not be told what the actual estimated correlation coefficient is.  This is a situation where the effect size (i.e. the estimated correlation coefficient) should be provided along with the inferential stuff.  I do agree that something that is called "size" should be magnitude-like and therefore not have a sign.Skbkekas (talk) 02:15, 7 February 2010 (UTC)
thread_title:  Cohen & r effect size interpretation 

message: Please note that it is unnecessary to change British spelling to American spelling. See Wikipedia:Manual of Style for more. — Chris53516 (Talk) 19:00, 8 June 2007 (UTC)
thread_title:  Spelling 

message: Should the text "As an estimator for the population effect size θ it is biased" actually read "g is biased"?
85.218.70.138 (talk) 12:33, 12 April 2010 (UTC)
replies: text: To me it seems that a large part of the Hedges g section of the present version is wrong. The Cohen's d is not heavily influenced by different sample sizes between the two groups compared to Hedges' g. Their forms for the pooled standard deviation are the same except for a "-2" in the denominator (at least in the presentation of Cohen's d that I got access to). — fnielsen (talk) 13:18, 8 October 2008 (UTC)
replies: text:  I have now erased a considerable part of the text. — fnielsen (talk) 14:18, 8 October 2008 (UTC)

text: I have now used the symbol g and g-star, instead of g-hat, so as not to make the confusion that g-hat is an estimator of g. — fnielsen (talk) 14:18, 8 October 2008 (UTC)

text: I am a bit daunted by some of the venom here as I'm not a statistician.  However, it seems to me that this has not been corrected. I don't have the Hedges & Olkin book but I do have the Hedges (1981) paper and the formula given there has the "-1" not "-9" and definitely approximates more closely to the correct value for g than the formula given here.  See my Rblog post: .  I am happy to change the formula to the correct one but worry that I'm missing something. Cpsyctc (talk) 12:43, 21 January 2024 (UTC)
thread_title:  Hedges G 

message: Currently, both Cohen's D and Hedge's G are presented in the same way: in both cases, the difference between the two means and the pooled standard deviations, with weights for S1 and S2 proportional to n-1 (n being the sample size). Then, what is the difference between the two? Borisba (talk) 15:43, 26 June 2022 (UTC)
thread_title:  What is the difference between Cohen's D and Hedge's G in the current definitions? 

message: Why does g have a hat? Neither in Hedges/Olkin nor Hartung/Knapp/Sinha is there a hat? — fnielsen (talk) 19:23, 7 October 2008 (UTC)
replies: text: I have now used the symbol g and g-star, instead of g-hat, so as not to make the confusion that g-hat is an estimator of g. — fnielsen (talk) 14:18, 8 October 2008 (UTC)
thread_title:  Hat? 

message: The equation for the correction factor had a -9 in the denominator instead of a -1. http://files.eric.ed.gov/fulltext/ED309952.pdf shows a -1 instead. If this equation was used to calculate any examples then they may need to be updated.Tdilorenzo (talk) 14:31, 1 June 2016 (UTC)
replies: text: I am a bit daunted by some of the venom here as I'm not a statistician.  However, it seems to me that this has not been corrected. I don't have the Hedges & Olkin book but I do have the Hedges (1981) paper and the formula given there has the "-1" not "-9" and definitely approximates more closely to the correct value for g than the formula given here.  See my Rblog post: .  I am happy to change the formula to the correct one but worry that I'm missing something. Cpsyctc (talk) 12:43, 21 January 2024 (UTC)
thread_title:  Correction Factor 

message: Jacob Cohen basically invented the concept of effect size. Or at least he brought it's importance to the forefront. That's why it's called Cohen's d. Why hasn't anyone referred to Jacob 'Jack' Cohen?  —Preceding unsigned comment added by 71.172.128.74 (talk) 01:47, 22 September 2008 (UTC)
replies: text: As of 7 october 2008 there are two references to his work. You can add more if you feel. — fnielsen (talk) 19:41, 7 October 2008 (UTC)
thread_title:  Why hasn't anyone cited Jacob Cohen? 

message: STATA which often seems the "industry standard" in carrying out meta-analyses uses slightly different equations from those used in this article. The program code for the METAN function may be found here  and uses the following equations:
Pooled standard deviation:
: s = \sqrt{\frac{(n_1-1)s^2_1 + (n_2-1)s^2_2}{n_1+n_2-2} }
i.e. difference is the -2.
The article says this equation is used for hedges g only, however it appears that the METAN function uses this version of the pooled standard deviation for Cohen's d, Hedges g and Glass's Delta. This equation is shown in the pooled standard deviation page too.
Variance of Cohens d (not yet explicitly in article):
: \hat{\sigma}^2(d) = \frac{n_1+n_2}{n_1 n_2} + \frac{d^2}{2(n_1 + n_2 - 2)}
Variance of Hedges g:
: \hat{\sigma}^2(g) = \frac{n_1+n_2}{n_1 n_2} + \frac{g^2}{2(n_1 + n_2 - 3.94)}
i.e. difference is the extra -3.94
Variance of Glass's &Delta (not yet explicitly in article):
: \hat{\sigma}^2(\Delta) = \frac{n_1+n_2}{n_1 n_2} + \frac{\Delta^2}{2(n_2 - 1)}
Does anyone know why these differences exist? 
Should we change the article to these equations?
194.83.139.137 (talk) 14:36, 30 January 2009 (UTC)
replies: text: Have we resolved this issue? I wrote the variance equation, but I am puzzled. The Hedges and Hartung books are not clear to me. I tried to derive the variance for Hedges g and g* myself and/but ended up with 2(n1+n2-4) in the denominator (asymptotically) for g*. — fnielsen (talk) 15:54, 8 November 2010 (UTC)
thread_title:  STATA meta-analysis equations are different from this article 

message: This section would some tidying up by someone with a suitable grasp of the details and authority from the scary editors. The sentence "Usually, μbaseline is zero, while not necessary. " in particular needs some work.
87.114.240.70 (talk) 20:50, 24 March 2010 (UTC)
thread_title:  T-test for mean difference of single group or two related groups 

message: I added the "expert" template, because from the section "Confidence interval and relation to noncentral parameters" onwards the English is so poor that it is difficult to see what is going on. There seems to be not even an attempt to explain what is going on. But there may be something important being said. This may relate to the immediately above comment. JA(000)Davidson (talk) 16:44, 27 May 2011 (UTC)
thread_title: Expert attention

message: Kelley has another paper out and defines effect size as a quantitative reflection of the magnitude of some phenomenon that is used for the purpose of addressing a question of interest. Might be worth encorporating into the article.  Agree with the sentiment above that the article needs work, by the way. Tayste (edits) 22:05, 12 September 2012 (UTC)
thread_title:  New paper on effect size in APA journal 

message: Is moderate the same as a medium effect size? If these can be used interchangeably, then I think the article should state this because both are common in the psychology literature. If there are other synonymous terms, I think they should be included as well. --1000Faces (talk) 22:05, 14 July 2013 (UTC)
thread_title:  Alternate names 

message: Hello everyone,
Perhaps Cohen's d is calculated incorrectly. The provided example might need clarification.
The text is:
"So, in the example above of visiting England and observing men's and women's heights, the data (Aaron,Kromrey,& Ferron, 1998, November; from a 2004 UK representative sample of 2436 men and 3311 women) are:
Men: mean height = 1750 mm; standard deviation = 89.93 mm
Women: mean height = 1612 mm; standard deviation = 69.05 mm
The effect size (using Cohen's d) would equal 1.72 (95% confidence intervals: 1.66 – 1.78). This is very large and you should have no problem in detecting that there is a consistent height difference, on average, between men and women."
I calculated Cohen's d from the provided means, standard devations, and group size. The value was 1.756.
When I tried to find the original paper presenting Cohen's d, I could not find the correct article. The link to Aaron, Kromrey, & Ferron (1998) does not seem to work in text, but the link in the reference list does seem to work. I could find the paper, but the data is not presented in that paper. Moreover, it is said that the data is from a 2004 UK sample. I cannot find which paper presented that data from the 2004 UK sample. In addition, I did not understand what is meant by "the example above".
All in all, it seems confusing to me. It leaves me with some questions:
1. What should be the correct value in the calculation example of Cohen's d?
2. Should a paper be cited with the data from the 2004 UK sample? Or if it already is in it somewhere, where can I find it?
3. What are the reasons to cite Aaron, Kromrey, & Ferron (1998) in this example and at that specific location?
4. What is meant by "the example above"?
Perhaps I misunderstood something. However, could anyone clarify this for me? Thanks!
Regards,
Joep  — Preceding unsigned comment added by 86.89.158.241 (talk) 15:14, 26 July 2013 (UTC)
thread_title:  Calculating cohen's d from example 

message: Under this article's section regarding "correlation coefficient", it says "... r², the correlation coefficient (also referred to as "r-squared")". However, the wiki page for correlation coefficient explains the term as referring to simply r. Moreover, [further down in that article] it is explained that r² is called the coefficient of determination. I do believe that the latter is the correct term for r². I might however be mistaken and so I hope someone more statistic savvy can confirm and correct this misnaming in the article. Thank you.
:Titles were corrected. Pearson r is the same as correlation coefficient. It can be used as an effect size although the coefficient of determination is also used. Dger (talk) 04:36, 29 June 2014 (UTC)
thread_title:  Coefficient of determination erroneously referred to as correlation coefficient? 

message: I've revised the first para., which currently starts with two definitions, gives an example of a phenomenon (not an effect size) then says that effect sizes are descriptive and not inferential but also inferential (WTF?) and fails to directly connect effect sizes to NHST. I added a recent, and I think compelling, example of how effect sizes should be used.Amead (talk) 05:25, 6 July 2014 (UTC)
thread_title:  Crappy beginning 

message: This article will not teach a layperson ANYTHING. It's incomprehensible. E.g. when someone reads "we got an effect size of 1.24" and comes here to understand what that means, they will end up NOT understanding what that means.
I first posted the above comment (as an edit summary) three years ago; the page has not improved in this regard.  My guess is that 95% of this page needs to be discarded as too technical (or moved to a wikibook on statistics), and the other 5% needs a complete rewrite for simplicity and clarity.  Gnuish (talk) 06:56, 12 January 2017 (UTC)
replies: text: Although Wikipedia is not a textbook, I'm sympathetic to this issue. I agree that articles that are incomprehensible to the average layperson are of limited value. But I doubt anything will come of these comments without listing specific things that are unclear. Also (and I'm not criticizing) your example isn't comprehensible to me either. It's like saying "the animal weighs 3 kg"... Without knowing what animal, I don't know if 3 kg is big (that would be a HUGE mouse) or tiny (like a micro elephant). An effect size of 1.24 is not interpretable without more information. If you tell me that's Cohen's d or Hedge's g, then I know it's a fairly large difference (about 1.25 standard deviations, which is a big difference for two variables distributed approximately normally). Finally, I'd like to point out that the reason this is all so complex is that we're describing things that generally have no natural metric, like temperature. So, in order to even begin to understand an effect size, you have to understand it's metric. For temperature, you have to be told whether it's Fahrenheit, Celsius, Kelvin, etc.... 35F, 35C, and 35K are really different temperatures.Amead (talk) 23:03, 25 August 2017 (UTC)
thread_title:  Incomprehensible article 

message: Hello fellow Wikipedians,
I have just modified one external link on Effect size. Please take a moment to review my edit. If you have any questions, or need the bot to ignore the links, or the page altogether, please visit this simple FaQ for additional information. I made the following changes:
*Added archive https://web.archive.org/web/20090323151854/http://www.mdp.edu.ar/psicologia/vista/vista.htm to http://www.mdp.edu.ar/psicologia/vista/vista.htm
When you have finished reviewing my changes, you may follow the instructions on the template below to fix any issues with the URLs.
Cheers.—InternetArchiveBot (Report bug) 06:02, 18 September 2017 (UTC)
thread_title:  External links modified 

message: Shouldn't Wilcoxon's r get a mention in the article? It is defined as
: r := \frac{z}{\sqrt{N}} 
where z is the z-score and N is the sample size or number of trials.
PedantNumber1 (talk) 18:28, 17 April 2022 (UTC)
thread_title:  Wilcoxon's r 

message: Many sources call it "Cohen's omega" ( \omega), not Cohen's w. The two letters look very similar, hence it's easy to mix them up, but they are different. In the Cohen's book (Statistical Power Analysis for the Behavioral Sciences, 1988), the letter is different from the regular latin w (in the book, it's denoted in bold, with a slightly different font), so I'd be inclined to think that in fact it's omega, not w. If I'm correct, this is something that should be corrected in the article. 85.169.195.108 (talk) 08:15, 12 March 2023 (UTC)
thread_title:  Cohen's w or omega?