DOCUMENT RESUME

ED 433 353

AUTHORTITLE
PUB DATENOTE

PUB TYPEEDRS PRICEDESCRIPTORS

TM 030 022

Aaron, Bruce; Kromrey, Jeffrey D.; Ferron, John
Equating "r"-based and "d"-based Effect Size Indices:
Problems with a Commonly Recommended Formula.
1998-11-03
13p.; Paper presented at the Annual Meeting of the Florida
Educational Research Association (43rd, Orlando, FL,
November 2-4, 1998).
Reports
Research (143)
Speeches/Meeting Papers (150)MF01/PC01 Plus Postage.
*Correlation; *Effect Size; *Estimation (Mathematics); *Meta
Analysis; Sample Size

ABSTRACT
Two general categories comprise the various effect size
indices that have been proposed for use in meta-analysis: (1) the "d"-type
estimator (based on magnitude of mean difference); and (2) the "r"-type
estimator (based on magnitude of correlation). In meta-analyses, researchers
often must convert these effect size indices to a common metric to aggregate
and synthesize results from various studies empirically. A commonly
recommended formula for equating mean-difference effect sizes andcorrelational effect sizes was found to lead to inaccurate results,
particularly with small sample sizes. A correct formula for converting"d"-based and "r"-based effect size indices is presented. Results of applying
the common and corrected formulas are illustrated for a variety of data
conditions at various sample sizes and effect sizes, suggesting that bias aslarge as 20 percent results from the common formula, something that can be
avoided by applying the alternative equation. (Author/SLD)********************************************************************************
Reproductions supplied by EDRS are the best that can be madefrom the original document.
********************************************************************************

Effect Size Indices1

Equating r-based and d-based effect size indices:
Problems with a commonly recommended formula

Bruce Aaron
Jeffrey D. Kromrey
John Ferron

University of South Florida

PERMISSION TO REPRODUCE AND
DISSEMINATE THIS MATERIAL
HAS BEEN GRANTED BY

U.S. DEPARTMENT OF EDUCATION

Office of Educational Research and Improvement

EDUCATIONAL RESOURCES INFORMATIONCENTER (ERIC)

Kis document has been reproduced asLireceived from the person or organization
originating it.
Minor changes have been made to
improve reproduction quality.

TO THE EDUCATIONAL RESOURCESINFORMATION CENTER (ERIC)

Points of view or opinions stated in thisdocument do not necessarily represent
official OERI position or policy.

A paper presented at the annual meeting of theFlorida Educational Research Association, Orlando, Florida, November 3, 1998

2

BEST COPY AVAILABLE

Effect Size Indices2

Abstract
Two general categories comprise the various effect size indices that have been proposed

for use in meta-analysis: (a) the d-type estimator (based on magnitude of mean difference), and (b)the r-type estimator (based on magnitude of correlation). In meta-analyses, researchers often must
convert these effect size indices-to a common metric to empirically aggregate and synthesize resultsfrom various studies. A commonly recommended formula for equating mean-difference effect sizes
and correlational effect sizes was found to lead to inaccurate results, particularly with small sample

sizes. A correct formula for converting d-based and r-based effect size indices is presented.
Results of applying the common and corrected formulas are illustrated for a variety of data
conditions at various sample sizes and effect sizes, suggesting that bias as large as 20% resultsfrom the common formula, all of which can be avoided by applying the alternative equation.

Effect Size Indices3

Equating r-based and d-based effect size indices: Problems
with a commonly recommended formulaPurposeThe purpose of this paper is to inform researchers about inaccuracies that can result fromapplying a commonly cited formula for converting effect size indices based on mean differences toeffect size indices based on correlation. A corrected formula for equating these d-based and rbased effect size indices is presented. In addition, a demonstration of the potential bias of thecommonly recommended formula is provided under data conditions displaying a variety of effectsizes, sample sizes, and degrees of inequality between sample sizes.

Background

Essential to synthesis of research are the processes of comparing and combining the resultsof studies pertaining to a particular research question. Meta-analysis is a prominent method ofsynthesis that uses statistical procedures to summarize findings from various individual studies,seeking an objective integration of the results of research conducted in an area of particularinterest. In recent years, meta-analytic methods for summarizing research results across studies
have proliferated in the social, behavioral, and biomedical sciences (Wolf, 1986).
Generally, meta-analysis summarizes independent statistical tests of a common hypothesis,

and the strength of the relationship of interest. The latter is summarized by calculating effect sizes.
Various effect size measures have been proposed for different test statistics (Rosenthal, 1994).
Two general categories of effect size are conceptualized as (a) the d-type estimator (based on
magnitude of mean difference), and (b) the r-type estimator (based on magnitude of correlation).

The d-type effect size estimators derive a standardized mean difference that Cohen (1988)
describes as a "pure number, one free of our original measurement_unit with which to index what
can be alternatively called the degree of departure from the null hypothesis of the alternative4

Effect Size Indices4

hypotheses .

.

.

". This difference between two sample means, expressed in standard deviation

units, is frequently referred to as Cohen's d , and is represented as:

d Xi X2(1)

Cr pooled

In correlational studies, an effect size can be described as a signal-to-noise ratio, andestimates the strength of relationship between interval or ratio scale measures. Such an effect size

estimate is offered by Cohen (1983, p. 117) as:

r

2

f2

2

1

(2)

r2

In meta-analytic reviews of literature, the set of studies defined for review by the
researcher often comprises diverse analysis procedures, requiring use of both d- and r-type

estimators. In order to aggregate and synthesize these results, one must translate the various effectsize estimates into a common metric. The statistic usually chosen as the basis for this common
metric is r (the Pearson Product-Moment coefficient) (Wolf, 1986). Procedures for converting d tor have been discussed by Cohen (1965, 1988), Friedman (1968), Glass, McGraw, and Smith(1981), Rosenthal (1984), and Wolf (1986), with the following formula provided for expressing d
as r:

r=

d2
(3)

d2 +4

Equivalently, a conversion from r to d is offered as:

d=

2
2r
(4)

k1;3:
5

Effect Size Indices
5

These formulas, however, negatively bias estimates of r relative to d, particularly whensample sizes are unequal. This was accounted for by Hedges and Olkin (1985, p. 77), who offered
the conversion from d to r appropriately as an approximation:

d2

r2

(5)

d2 + (ni + n22)/nwhere n = (nin2) (ni + n2) . When nl= n2 = n formula 5 reduces to:

r2

d2

d2

d'i-4(n -1)/n

d2 +4

(6)

This indicates correctly both the approximation provided by formula 3, and the conditional
appropriateness of formula 3 for conversion when sample sizes are equal.
One way of illustrating the relationship of d- and r- based estimators is by considering thetest of significance for differences between two sample group means, where the t-test of thestandardized mean differences between groups, and the t-test associated with the point biserial
correlation between group and dependent variable, are identical. An exact equation for theconversion of d to r can be derived from the relationship of the test of significance and effect size

for a given study, noted by Rosenthal (1994) as
Test Statistic = Effect Size x Size of Study,or, for the specific case in which effect size is expressed as the standardized mean difference
between two groups,

t

XI X2
Sp

x

X2

1

1n1 +

1

1

(7a)
1

I

P r nin2Effect Size Indices6

It follows that
(7b)

X2

t=

1 -r2

spVT1; +

d= r

[11

1- r2

(7c)

1+

1

ni

n2

d2= r2(N 21 [1 +1- r2)

d2 d2 r2

ni

+

n2
1

r2 (N 2) (1

1 )1
n2

nl

(7d)

n2

n1

(1- r2)d2 = r2 (N1d21

d2= r2[(N_ 2) (I+ I)

(7e)
(70

n2

(7g)

n2

(7h)

ni

d2

r2
d 2 + (N

+
n1

d2

(8)

d2 N2 2Nnin2
Equation 8 can be seen as an alternative derivation of a conversion formula offered by
Cohen (1988, p. 24) for the general case (i.e., applicable to unequal sample sizes), withp as the
proportion in one group on a dichotomous variable, and q = 1 -p:

r2

d2

(9)

d2 +(jq -)

P

(9a)

d2

d2 + N2

nin2

7

Effect Size Indices
7

Therefore, corrected equation 8 can be seen to differ from equation 9a by replacement of
N2 with N2 -2N in the second term of the denominator.
For balanced situations, in which n1 = n2, corrected equation 7h reduces as follows:r2d2d2-F(N-2)()
N

(10)

d2

d2 + 4 (L

r=

(10a)

d2

d2 +4 -II

(10b)

Thus, for balanced situations, corrected equation 10b is seen to differ from commonly
recommended formula 3 by subtraction of the last term in the denominator.

Data Source and Method
To provide a demonstration of the bias associated with formulas 3 and 9, the equations
were applied to data conditions for pairs of samples having varying effect sizes, sample sizes, anddegrees of inequality between sample sizes. These conditions represented sample pairs withstandardized mean differences (d - type effect sizes) of 0.2, 0.5, and 0.8. These levels represent

treatment effects ranging from small to large, as suggested by Cohen (1988). Sample sizes forcases in which ni= n2= n were 5, 10, 20, 30, 40, 50, and 100. For the unbalanced sample size
comparisons, n1 + n2= 20, and the magnitude of inequality between sample sizes ranged from zero

(where nl= n2= 10), to a ratio of 9:1 (where n1=18 and n2=2). These conditions were deemed toprovide a practical range for investigating the differences resulting from application of thecommonly recommended formulas for equating d-type and r-type effect size estimators andapplication of the alternative corrected formulas.

Effect Size Indices
8

Values of r derived by the commonly recommended formulas (formulas 3 and 9) and thevalues of r provided by the corrected formulas (formulas 8 and 10b) were calculated for eachunique set of conditions. These values of r were then used to calculate f2. Bias is shown as thedifference between the tabled values of r and f 2 derived by the common and actual formulas. The
proportion of the actual effect size estimate comprising this bias is shown as proportional bias.

Results and Conclusions
Results presented in tables 1 and 2 demonstrate the magnitude of discrepancy found when
applying the commonly recommended formula, and the alternative equations shown as formulas 8

and 10, for balanced and unbalanced sample sizes. Differences between the tabled values of r andf 2 derived by the commonly recommended formulas, and the values of these effect size estimates asprovided by the corrected formulas, are shown as bias, with proportional bias indicating the size of
this bias as a proportion of the actual effect size estimate (calculated with the corrected formula).

As shown in tables 1 and 2, bias associated with the common formula is particularly
problematic for small sample sizes and when converting a d-based effect size estimate to an f2-

based effect size estimate. Among the conditions examined, the largest bias (20%) was foundwhen translating d to f 2 for nl= n2 = 5. Even for n1= n2 = 100, a bias of 1% was avoided with thecorrected formula. With unbalanced samples, conversion of d to f 2 incurred a bias of 10%, andestimation of r from d biased effect size estimates by 4% to 5%. For unbalanced samples, theextent of inequality in sample sizes had no practical effect on proportional bias. When estimatingf 2 from d for these conditions, this bias was constant at 10%. The size of treatment effectdemonstrated no practical influence on the extent of bias found for estimates of either f 2 or r, given

any particular combination of n, and n2. Given these results, use of the corrected formulas byanalysts seeking to synthesize d-type and r-type estimates in meta-analysis is recommended.

Effect Size Indices9

Limitations

The inaccuracy resulting from application of the more commonly cited, approximate
equation will differ according to the range of n, and might be negligible for certain conditions. For
example, less bias was found for larger samples and when estimating r from d. Discrepancies
tended to be negligible, therefore, when estimating r from d with balanced sample sizes of 50 or
more.

Implications

In meta-analyses, researchers often encounter studies utilizing diverse research designs inwhich some effect sizes might be based on standardized differences in means, while others might bebased on the correlation between variables. In order to empirically aggregate and synthesize theresults from such diverse analyses, the meta-analyst must convert these effect sizes to a common
metric. The commonly recommended formulas for equating correlational effect sizes and meandifference effect sizes were found to lead to inaccurate results, particularly for small samples andwhen translating d to f 2. Corrected formulas for converting d-based and r-based effect size indiceswere presented and demonstrated. Since the corrected formulas are straightforward, and theaccuracy of meta-analysis can be improved by their use, this is an important detail for researchers
concerned with meta-analytic methods.

Table 1. Estimations of r and f Z with equal sample sizes
Estimating r from dCommon Formula Actual (Corrected Formula)

N1

N2

d

5
10

5

0.2
0.2
0.2
0.2
0.2
0.2
0.2

.0995
.0995
.0995
.0995
.0995
.0995 _
.0995

.1111
.1048
.1021
.1012
.1008
.1005
.1000

Bias *- .0116- .0053- .0026- .0017- .0013- .0010- .0005

Proportional Bias **_.
- .1045- .0508

0.5
0.5
0.5
0.5
0.5
0.5
0.5

.2425
.2425
.2425
.2425
.2425
.2425
.2425

.2692
.2548
.2485
.2464
.2454
.2449
.2437

- .0267- .0123- .0059- .0039- .0029- .0023- .0011- .0990- .0482- .0238- .0158- .0118- .0095- .0047

0.8
0.8
0.8
0.8
0.8
0.8
0.8

.3714
.3714
.3714
.3714
.3714
.3714
.3714

.4083
.3885
.3797
.3768
.3755
.3746
.3730

- .0368- .0171- .0083- .0055- .0041- .0032- .0016- .0903- .0441- .0218- .0145- .0108- .0087- .0043

10

20
30
40
50

20

100

100

5
10

10

30
40
50

5

20
30

20
30
40
50
100

100

5
10

5
10

20
30
40
50
100

20
30
40
50

40
50

100

[::::::::::::::::::::::;:::::::§:t::::::::::::;:gi.i:::.:

5
10

5
10

20
30
40
50

20
30
40
50

100

100

5

5

10

10

20
30
40
50
100

20
30
40
50

5

10

100
5
10

20
30
40
50

40
50

100

100

20
30

:::ii:i: i:i::::::i:::..

:""iqi:::::::::::::::::::::- .0251- .0166- .0125- .0100- .0050

Estimating [2 from d

0.2
0.2
0.2
0.2
0.2
0.2
0.2

.0100
.0100
.0100
.0100
.0100
.0100
.0100

.0125
.0111
.0105
.0103
.0103
.0102
.0101

- .0025- .0011- .0005- .0003- .0003- .0002- .0001- .2000- .1000- .0500- .0333- .0250- .0200- .0100

0.5
0.5
0.5
0.5

0.5
0.5

.0625
.0625
.0625
.0625
.0625
.0625
.0625

.0781
.0694
.0658
.0647
.0641
.0638
.0631

- .0156- .0069- .0033- .0022- .0016- .0013- .0006- .2000- .1000- .0500- .0333- .0250- .0200- .0100

0.8
0.8
0.8
0.8
0.8
0.8
08

.1600
.1600
.1600
.1600
.1600
.1600
1600

.2000
.1778
.1684
.1655
.1641
.1633
1616

- .0400- .0178- .0084- .0055- .0041- .0033- .2000- .1000- .0500- .0333- .0250- .0200- 0016- 01000.5.

Note: All estimates rounded to four digits
Bias = (Conunon -Actual)** Proportional Bias = (Bias ÷ Actual)
*

11

IP

-

=

=

=

=

,I

AM

. ..

.

'

-

AMP

I1,

=

=

-

4..

d

a

-

--

a

ID M

.

4111D

M1.

0

Q.,

CEEEEEEEEEIEEEEEEEEEIEEEEEEEEE EEEEEEEEEEEEEEEEEEEEEEEEEEE'

moconsionommen socionsouncono-commensectimec menclemon 1 - : .

:

..

GA

I

-

;

:

References

Wolf, F.M. (1986). "Meta-analysis: Quantitative methods for research synthesis". Sage
University Paper series on Quantitative Applications in the Social Sciences, 07-059. Beverly
Hills: Sage Pubns.
Rosenthal, R. (1994). Parametric measures of effect size, In H. Cooper & L.V. Hedges,(eds.), The handbook of research synthesis (pp 232-244).
Cohen, J. and Cohen P. (1983). Applied Multiple Regression/Correlation Analysis for the
Behavioral Sciences (2nd Ed.). Hillsdale, NJ: LEA.
Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.).
Hillsdale, NJ: LEA.

Friedman, H. (1968). Magnitude of experimental effect and a table for its rapid
estimation. Psychological Bulletin 70:245-251.

Glass, G.V., McGraw, B., and Smith M.L. (1981). Meta-analysis in social research.
Sage: Beverly Hills.
Hedges, L.V. and 011cin, I. (1985). Statistical methods for meta-analysis. Academic
Press: Orlando.

Rosenthal, R. (1984). Meta-analysis procedures for social research.. Sage: Beverly
Hills.

IC

U.S. Department of Education
Office of Educational Research and Improvement (OERI)National Library of Education (NLE)
Educational Resources Information Center (ERIC)

TM030022

REPRODUCTION RELEASE(Specific Document)

I. DOCUMENT IDENTIFICATION:
66..se8

Est/A.m.-4- irt,

Tile:

6asea e:c.CeA

Arid.

Protevv-s W t-EG-, ek- ea fvr."-e) u t C`t recon, pykeirk jed'
Author(s):tjrince.

b-5-e

AcAirGrlz_e fh Ices;---CoroAu tk.

ivrt cerro

1Ar
Publication Date:

Corporate Source:

Ak(AInTr..rel-'5[4r--/01-t-etfet

/tit

I (

II. REPRODUCTION RELEASE:
In order to disseminate as widely as possible timely and significant materials of interest to the educational community, documents announced In themonthly abstract joumal of the ERIC system, Resources in Education (RIE), are usually made available to users in microfiche, reproduced paper copy,
and electronic media, and sold through the ERIC Document Reproduction Service (EDRS). Credit is given to the source of each document, and, if
reproduction release is granted, one of the following notices is affixed to the document.
If permission is granted to reproduce and disseminate the identified document, please CHECK ONE of the following three options and sign at the bottomof the page.
The sample sticker shown below will be
Waxed to all Level 1 documents

The sample sticker shown below will be
anted to ail Level 2A documents

The sample sedter shown below will be@tided to all Level 2B doannents

PERMISSION TO REPRODUCE AND
DISSEMINATE THIS MATERIAL HAS
BEEN GRANTED BYPERMISSION TO REPRODUCE AND
DISSEMINATE THIS MATERIAL IN
MICROFICHE, AND IN ELECTRONIC MEDIAFOR ERIC COLLECTION SUBSCRIBERS ONLY,
HAS BEEN GRANTED BYPERMISSION TO REPRODUCE AND
DISSEMINATE THIS MATERIAL IN
MICROFICHE ONLY HAS BEEN GRANTED BY

TO THE EDUCATIONAL RESOURCESINFORMATION CENTER (ERIC)TO THE EDUCATIONAL RESOURCESINFORMATION CENTER (ERIC)\ecPs
TO THE EDUCATIONAL RESOURCESINFORMATION CENTER (ERIC)

2B

2A

Level 2B

Level 2A

E
Check here for Level 1 release. permitting reproduction
and dissemination in rnicrotiche or other ERIC archival
media (e.g.. electronic) and paper copy.

Chedt here for Level 2B release, permitting
reproduction and dissemination in microfiche only

Check here for Level 2A MINIM, pemtiteng reproduction
and dissemination In rnialsache and in electronic media
for ERIC midievei collecbon subsatbers only

Documents will be processed as indicated provided reproduction quality pelmets.
If permission to reproduce is wanted. but no box is checked. documents will be processed at Level 1.

I hereby grant to the Educational Resources Information Center (ERIC) nonexclusive permission to reproduce and disseminate this document

as indicated above. Reproductio'n from the ERIC microfiche or electronic media by persons other than ERIC employees and its system
contractors requires permission from the copyright holder. Exception is made for non-profit reproduction by libraries and other service agencies
s of educators in response to discrete Inquiries.
to satisfy information
PrnitediMmeiPositioniTitle:Sign

here,-i
pleasegeOrganciatcriAddress:

Aind?rAee VIC M3411-14k

5+, dnc,,rek

C-'1411-1

tUie,)

A-Abfarn

Telephone:

60o t-1111( 665'7

E-Mail Address:

irarkter AO( ,ron

p

,
FAX

VA I /A Ate it

Date:6, /7.

rq(over)

III. DOCUMENT AVAILABILITY INFORMATION (FROM NON-ERIC SOURCE):
If permission to reproduce is not granted to ERIC, or, if you wish ERIC to cite the availability of the document from another source, pleaseprovide the following information regarding the availability of the document. (ERIC will not announce a document unless it Is publicly
available, and a dependable source can be specified. Contributors should also be aware that ERIC selection criteria are significantly more
stringent for documents that cannot be made available through EDRS.)

Publisher/Distributor:

Address:

Price:

IV. REFERRAL OF ERIC TO COPYRIGHT/REPRODUCTION RIGHTS HOLDER:
If the right to grant this reproduction release is held by someone other than the addressee, please provide the appropriate name andaddress:
Name:

Address:

V. WHERE TO SEND THIS FORM:
Send this form to the following ERIC Clearititse:vERsiTy

MARYLANDERIC CLEARINGHOUSE ON ASSESSMENT AND EVALUATION
1129 SHRIVER LAB, CAMPUS DRIVE
COLLEGE PARK, MD 20742-5701
Attn: Acquisitions

However, if solicited by the ERIC Facility, or if making an unsolicited contribution to ERIC, return this form (and the document being
contributed) to:

ERIC Processing and Reference Facility
1100 West Street, 2" Floor
Laurel, Maryland 20707-3598
Telephone: 301 -497.4080
Toll Free: 800-799-3742
FAX: 301-953-0263

e-mail: ericfac@ineted.gov
WWW: http://ericfac.piccard.csc.com
EFF-088 (Rev. 9/97)
PREVIOUS VERSIONS OF THIS FORM ARE OBSOLETE.