Package: effectsize
effectsize: computing effect sizes in Pythoneffectsizeis a comprehensive Python package for computing effect sizes (ESs), also known as standardized differences. The package implements the methodology outlined byYang and Dalton (2012)and it provides complex functionality, such as the ability to deal with skewed variables, multinomial categories, and weighted statistics.PrerequisitesBefore you begin, ensure you have installed Python 3.7 or highereffectsizehas four dependencies:numpy,pandas,scipy,statsmodelsInstallationBinary installers for the latest released version are available at thePython Package Index (PyPI):pipinstalleffectsizeAs well as through the conda-forge channel onConda:condainstall-cconda-forgeeffectsizeSource code foreffectsizeis hosted on itsGitHub repositoryUsageTo useeffectsize, it must first be imported as a Python module:importeffectsizeFrom here, all ofeffectsize's functionality is accessible through a single function namedcompute, which is called via.effectsize.compute(). This function takes up to 8 arguments, which are outlined below along with their default values:effectsize.compute(data,group,continuous=[],categorical=[],skewed=[],weights=None,decimals=2,intervals=None)Given aPandas DataFrameand a variable specifying 2 groups,effectsize.compute()will return anotherPandas DataFramecontaining ESs for all variables that are requested by the user. Detailed description for each argument ofeffectsize.compute()is presented below:data(Pandas DataFrame): Each row should be an observation and each column should be a variable. In other words, all variables for which the user would like to compute an ES must be a column within the DataFrame.group(str): This should be the variable defining the two groups, specified as a string. Ideally, these two groups should be coded as 0 (control) and 1 (treatment), buteffectsizewill work regardless of the coding system used, provided it specifies two groups. Note that if the coding is switched then, the sign of the ES for continuous variables will be reversed, but the magnitude will stay the same. This is typically not an issue as the direction can be inferred from summary statistics.continuous(list): This should contain the names of all of the continuous variables for which the user would like an ES computed. This must be specified as a list containing the variable names as strings e.g.,continuous = ["age", "salary", "bmi"]would be syntactically correct butcontinuous = [age, salary, bmi]would not. If there are no continuous variables for which an ES needs to be computed, thencontinuousshould be passed an empty list, which is also the default object passed to the argument.categorical(list): This should contain the names of all of the categorical variables for which the user would like an ES computed. In the exact same way as thecontinuousargument, this must be passed a list containing the variable names as strings. If there are no categorical variables for which an ES needs to be computed, thencategoricalshould be passed an empty list, which is also the default object passed to the argument.skewed(list): This should contain the names of all of the continuous variables which have a skewed distribution, for which the user would like an ES computed. Note that the skewed variables must be specified in both thecontinuousargument and theskewedargument. For example, if age follows a skewed distribution, then this should be specified aseffectsize.compute(..., continuous = ["age", "salary", "bmi"], skewed = ["age"]). If this were to be specified aseffectsize.compute(..., skewed = ["age"]), then the age variable will simply be ignored and no ES returned. In the exact same way as thecontinuousargument, this must be passed a list containing the variable names as strings. If there are no skewed variables for which an ES needs to be computed, thenskewedshould be passed an empty list, which is also the default object passed to the argument.weights(Noneorstr): This should be the variable defining weights, specified as a string (examples of weights include sampling weights or propensity scores). Note that the sum of all of the weights must be >= 1, else the computed ES will not be correct. If there are no weights, thenweightsshould be passed the valueNone, which is also the default value passed to the argument.decimals(int): This should be an integer which specifies the number of decimals to which the ESs should be computed, the default value is 2.intervals(Noneorfloat): This should be a value between 0 and 1 specifying the level of confidence interval (CI) which the user would like e.g., to compute a 95% CI, this should be specified asintervals = 0.95. Note if CIs do not need to be computed thenintervalsshould be passed the valueNone, which is also the default value passed to the argument.effectsizeexcludes all observations for which data is missing ongroup(i.e., it is not clear to which of the 2 groups the observation belongs), or if data is missing on the variable for which the user would like ESs computed (i.e., those incontinuousand/orcategorical). Therefore, it is advised that users deal with missing data in the most appropriate manner for their analyses prior to computing ESs.The order in which the ESs appear in the output ofeffectsize.compute()is the same order in which the variables appear in theDataFramepassed todata. This is to ensure consistency between the output ofeffectsizeand the user's originalDataFrame. It does not matter in which order users specify the variable names inside ofcontinuousandcategorical, the results will always be output so that they correspond to the same order as the originalDataFrame.Simulation examplesTo demonstrate examples of how to useeffectsize, we simulated 2 groups, each containing 100 observations. In each group, we simulated 4 variables of interest:var1is a Normally distributed continuous variable,var2is an exponentially ditribusted (i.e., skewed) continuous variable,var3is a 2-level categorical variable, andvar4is a 3-level categorical variable. We used different parameter values to ensure that the distributions of the variables were different between the 2 groups. Summary statistics for the simulated dataset are presented inTable 1.Table 1.Summary of simulated data. Continuous variables are presented as mean (standard deviation) and categorical variables are presented asn(%).VariableGroup 1 (n= 100)Group 2 (n= 100)var11.02 (0.7)1.15 (0.9)var20.37 (0.4)0.30 (0.3)var3Level 081 (81%)65 (65%)Level 119 (19%)35 (35%)var4Level 024 (24%)32 (32%)Level 146 (46%)38 (38%)Level 230 (30%)30 (30%)We will assume that thePandas DataFramein which these data are stored is nameddf, and the name of the variable specifying the group to which each observation belongs is namedgroup.To compute ESs for the continuous variables only:effectsize.compute(data=df,group="group",continuous=["var1","var2"])ESvar10.16var2-0.20However we knowvar2is skewed, so we may want to account for this:effectsize.compute(data=df,group="group",continuous=["var1","var2"],skewed=["var2"])ESvar10.16var2-0.11We see a small change in the ES forvar2after accounting for the fact that it is skewed. We can also compute ESs for the categorical variables only:effectsize.compute(data=df,group="group",categorical=["var3","var4"],)ESvar30.37var40.20Finally, if we wish to compute ESs for all variables at once:effectsize.compute(data=df,group="group",continuous=["var1","var2"],categorical=["var3","var4"],skewed=["var2"])ESvar10.16var2-0.11var30.37var40.20Obtraining extra precision is also straightforward:effectsize.compute(data=df,group="group",continuous=["var1","var2"],categorical=["var3","var4"],skewed=["var2"],decimals=4)ESvar10.1632var2-0.1150var30.3664var40.1961Conifdence intervals are easily computed, in this case 95% CIs:effectsize.compute(data=df,group="group",continuous=["var1","var2"],categorical=["var3","var4"],skewed=["var2"],intervals=0.95)ES95.0% CIvar10.16[-0.12, 0.44]var2-0.11[-0.39, 0.17]var30.37[0.09, 0.65]var40.20[-0.08, 0.48]Similarly, for 99% CIs:effectsize.compute(data=df,group="group",continuous=["var1","var2"],categorical=["var3","var4"],skewed=["var2"],intervals=0.99)ES99.0% CIvar10.16[-0.20, 0.52]var2-0.11[-0.47, 0.25]var30.37[0.00, 0.74]var40.20[-0.17, 0.57]We then create simulated weights for the observations by taking 200 samples from a Normal distribution with mean = 100 and standard deviation = 15. The variable containing the weights is namedwgt, and we can compute a weighted ES by specifying this in the function call:effectsize.compute(data=df,group="group",continuous=["var1","var2"],categorical=["var3","var4"],skewed=["var2"],weights="wgt")ESvar10.20var2-0.14var30.40var40.19Empirical examplesTo demonstrate use on a real-world example, we use data from the 2017-2018National Health and Nutrition Examination Survey (NHANES), a cross-sectional study which samples the noninstitutionalized US population. Participants are first interviewed at home and then invited to a mobile examination center for further interviews, tests, and examinations. We will use data from NHANES to evaluate differences between smokers versus non-smokers, across 6 variables:Age (in years)Sex (male / female)Ethnicity (White / Black / Hispanic / Other)Education (below high school / high school graduate / college graduate)BMI (in kilograms per square metre)Blood cholesterol (in milligrams per deciliter)Age, BMI, and blood cholesterol were measured as continuous variables, whilst sex, ethnicity, and education were measured as categorical variables. Summary statistics for the distribution of the variables amongst smokers and non-smokers are presented inTable 2.Table 2.Summary of NHANES data. Continuous variables are presented as mean (standard deviation) and categorical variables are presented asn(%).Non-smokers (n= 3981)Smokers (n= 870)Age52.2 (17.9)47.5 (15.6)SexMale1836 (46.1%)495 (56.9%)Female2145 (53.9%)375 (43.1%)EthnicityWhite1334 (33.5%)371 (42.6%)Black841 (21.1%)254 (29.2%)Hispanic999 (25.1%)124 (14.3%)Other807 (20.3%)121 (13.9%)EducationBelow high school751 (18.9%)205 (23.6%)High school graduate2155 (54.1%)580 (66.7%)College graduate1075 (27.0%)85 (9.8%)BMI30.0 (7.3)29.2 (7.8)Cholesterol188.0 (40.8)189.6 (43.5)We will assume that thePandas DataFramein which these data are stored is namednhanes, and the name of the variable specifying whether indivduals are smokers or non-smokers is namedsmoking.To compute ESs for all variables:effectsize.compute(data=nhanes,group="smoking",continuous=["age","BMI","cholesterol"],categorical=["sex","ethnicity","education"])ESage-0.28sex0.22ethnicity0.37education0.46BMI-0.12cholesterol0.04On further investigation, we may find that BMI is not distributed symmetrically and want to account for this:effectsize.compute(data=nhanes,group="smoking",continuous=["age","BMI","cholesterol"],categorical=["sex","ethnicity","education"],skewed=["BMI"])ESage-0.28sex0.22ethnicity0.37education0.46BMI-0.15cholesterol0.04In NHANES, probability sampling weights are used and stored in a variable namedwtmec2yr. If we wish to account for these sampling weights:effectsize.compute(data=nhanes,group="smoking",continuous=["age","BMI","cholesterol"],categorical=["sex","ethnicity","education"],skewed=["BMI"],weights="wtmec2yr")ESage-0.32sex0.11ethnicity0.17education0.56BMI-0.18cholesterol0.02The level of precision and CIs can be modified within thedecimalsandintervalsarguments, as demonstrated in earlier simulation examples.Finally, to demonstrate the effect of how the group coding, we switched how smokers and non-smokers were coded. In the examples above, smokers were coded as 1 and non-smokers as 0. We now re-run the final example from above, but having switched smokers to 0 and non-smokers to 1:effectsize.compute(data=nhanes,group="smoking_switched",continuous=["age","BMI","cholesterol"],categorical=["sex","ethnicity","education"],skewed=["BMI"],weights="wtmec2yr")ESage0.32sex0.11ethnicity0.17education0.56BMI0.18cholesterol-0.02We see that the magnitude of the ESs has remained unchanged, but the direction for the continuous variables has reversed. By referring toTable 2, we can see that the smokers tend to be younger, have a lower BMI, and have a higher blood cholesterol. Therefore, if smokers are our reference group, then we would expect the ESs for age and BMI to be negative, whilst the ES for cholesterol would be positive. However, if we were to take non-smokers as our reference group, then we would expect the ESs for age and BMI to be positive, whilst the ES for cholesterol would be negative, explaining the change in sign of the ESs in the above example. ESs for the categorical variables are always positive as computing them involves taking squares of matrices, which will yield positive values.ContributingUsers are actively encouraged to test and implementeffectsizein their projects, as well as leave feedback and make contributions to the package. In particular, we welcome contributions relating to improving computational efficiency, adding features which are likely to be widely used, and developing the underlying mathematical theory. Users canfork the softwareandcreate pull requestson GitHub, or get in touch regarding any relevant developments in statistical theory.ContactIf you wish to contact me you can reach me atnbashir562@gmail.comLicenseMIT License