A Practitioner’s Guide To Difference-In-Differences Approach | by Leihua Ye, PhD | Towards Data Science

Executive Summary
In a perfectly competitive market, conventional economic theory predicts an increase in the minimum wage leads to a rise in unemployment.
Cross-sectional studies provide conflicting evidence but not a causal effect.
Card and Krueger adopt a DID approach and find the wage increase leads to higher employment.
Foreword
In this post, I’ll replicate Card and Krueger’s paper with step-to-step explanations and complement with additional information on the definition of quasi-experimental design, assumptions, R code, and a Potential Outcomes Framework explanation.
What is a quasi-experiment?
A quasi-experimental design is sort of an experimental design without a true randomization process. Due to the lack of a random assignment, the treatment and control groups are not equivalent before the intervention. So, any differences from these two groups could be caused by the pre-existing differences.
The quasi design has been well received by policy analysts, health care providers, and Direct-To-Consumer companies (in particular, Netflix and Airbnb).
Why a quasi-experimental design?
In other posts, I’ve explained the benefits of experimental studies and why they should be the number one choice for causal inference, if possible. So, why don’t we run experiments all the time?
Great question! A quick answer is experimental designs are not good for some types of problems. For example, it’s not ethical to randomly assign people to hazardous conditions (e.g., smoking) that may harm their health.
In the policy arena, we can’t randomly assign California, not New York, to receive a new tax break.
Besides, there are spillover effects between the treatment and control groups in experimental designs. Netflix doesn’t run A/B testing for some types of questions but rely on quasi-experimental solutions (In this post, Netflix explained why A/B testing isn’t feasible all the time).
Under these circumstances, the quasi designs come in handy when a full-fledged experimental study isn’t an option.
“The results of a quasi-experiment won’t be as precise as an A/B, but we aspire towards a directional read on causality.”
— — Colin McFarland, Michael Pow, Julia Glick @ Netflix
Background
A bill signed into law on November 1989 raised the federal minimum wage from $3.35 per hour to $3.8 effective April 1, 1990, with a further increase from $4.25 per hour to $5.05 on April 1, 1992. New Jersey experiences the wage increase but not in Pennsylvania. David and Krueger run two rounds of survey right before and after the rise.
A brief timeline that may help:
February and March 1992: initial round of surveyApril 1992: an increase in the minimum wage from $4.25 to $5.05 in New JerseyNovember and December 1992: a follow-up survey
February and March 1992: initial round of survey
April 1992: an increase in the minimum wage from $4.25 to $5.05 in New Jersey
November and December 1992: a follow-up survey
Here are the reasons why we run quasi design. #1 New Jersey and Pennsylvania are not comparable in a lot of aspects. #2 The legislative process isn’t random.
However, we can identify the causal effects of the wage increase on the unemployment rate following a DID approach.
A quasi-experiment design does not require equivalent groups.
A Potential Outcomes Framework Analysis of the study
Here are the notations. Let D represent whether it receives the treatment, T for the time, i for the number of research subjects.
Di = 1: New Jersey receives the treatment (minimum wage increase)
Di = 0: Pennsylvania does not receive the treatment
T=1: After the wage increase (1992 November and December)
T=0: Before the wage increase (1992 February and March)
Y: the outcome variable (unemployment rate)
There is a potential problem. We don’t know how the control group (Pennsylvania) would behave if they would have received the treatment (missing E[Y0i(1)|D=1]).
We can use the DID approach can solve this issue:
the ATT E[Y1(1) — Y(0(1)|D=1] =
{E[Y(1)|D=1] — E[Y(1)|D = 0]} — {E[Y(0)|D=1] — E[Y(0)|D=0], assuming parallel trends.
The Parallel Trends Assumption means the difference between the treatment and control groups would stay constant if there were no treatment. In this case, the differences between the employment rates in NJ and PA would be the same if there were no wage increase law.
R analysis:
# Credits to Jonathan Mummolod <- read.csv(“wage_data.csv”,convert.factors = FALSE)head(d[, c(“ID”, “nj”, “postperiod”, “emptot”)])with(d,  ( mean(emptot[nj == 1 & postperiod == 1], na.rm = TRUE) — mean(emptot[nj == 1 & postperiod == 0], na.rm = TRUE)  )- (mean(emptot[nj == 0 & postperiod == 1], na.rm = TRUE) — mean(emptot[nj == 0 & postperiod == 0], na.rm = TRUE) ))ols <- lm(emptot ~ postperiod * nj, data = d)coeftest(ols)
To control for region and time differences, we can build a fixed effects regression model with unit and period fixed effects.
#A set of estimators and tests for panel data econometricslibrary(plm)library(lmtest)d$Dit <- d$nj * d$postperiodd <- plm.data(d, indexes = c("ID", "postperiod"))did.reg <- plm(emptot ~ postperiod + Dit, data = d,model = "within")coeftest(did.reg, vcov=function(x)#Heteroskedasticity-Consistent Covariance Matrix EstimationvcovHC(x, cluster="group", type="HC1"))
After comparing it to the original table, we have successfully replicated the results!
The original paper is here.
The replication dataset is here. A caveat: this is the raw data, and we need to process it for analysis.
Medium recently evolved its Writer Partner Program, which supports ordinary writers like myself. If you are not a subscriber yet and sign up via the following link, I’ll receive a portion of the membership fees.
leihua-ye.medium.com
Please find me on LinkedIn and Youtube.
Also, check my other posts on Artificial Intelligence and Machine Learning.