Causal Inference via CausalImpact | by Pranav Prathvikumar | Towards Data Science

Wikipedia defines it as the process of drawing a conclusion about a causal connection based on the conditions of the occurrence of an effect. In simpler words, Causal Inference is about determining the impact of an event/change on the desired outcome metric. Some instances of this being, if a newly launched marketing event has had an impact on the sales of the company or whether a new law has had an impact on how people behaved. While looking at graphs and comparing pre and post periods can give a good idea, but in most cases, this is simply not good enough. Especially in cases where there is a lot at stake, such as a multimillion-dollar marketing budget. Determining the causal impact of an event becomes even more critical in the fields of medicine or public policy, where such decisions can impact millions of people.
Suppose there was an event/treatment that took place, such as a marketing event. Let's represent it by this pill:
By observing we know that this is the outcome after the treatment :
What is also important is to know what would have happened if the event did not take place. Let’s represent the outcome that would have happened otherwise with this:
Now by comparing the two possible outcomes we can infer or measure the impact of the event. We can say that the customer being happy instead of sad is the impact of the treatment.
But the question in the real world is, how do we do that? How can we simultaneously know what happened and what might have happened? In such a case how do we determine the causal inference of such an event?
Let’s make it more clear by tabulating it and comparing the various scenarios:
Let there be two people, 1 and 2, and they are both being exposed to the treatment/event. By exposing them both to the treatment we can also observe the outcomes presented. But then we do not know what the possible outcomes otherwise might have been.
As it is not possible to know it, the next best thing we can do is estimate the other possible outcome.
A randomized controlled trial is a scientific experiment for determining the impact of a treatment on the desired outcomes. This is usually employed in the medical fields and is considered as the gold standard. In an RCT people are randomly selected and divided into two groups, where one is exposed to the treatment and the other is not. Due to the randomness involved, both groups are considered similar and any differences in the outcomes can be attributed to the difference in treatments.
Again, tabulating the above for a clearer understanding:
Now there are four people, randomly selected and divided. Half are exposed to the treatment and the other half is not. The potential outcomes of the two groups are estimated based on the outcome of the other group.
This estimated outcome is what is often known as a counter factual.
While RCT is the best possible way to estimate causal inference, it might not be possible in all cases. Maybe conducting an RCT is too expensive or too difficult. For instance, it might be too difficult to conduct an RCT in a small country as it is difficult to separate into test and control. Or maybe a RCT was not done at all and a retrospective study is now being conducted to understand the impact.
In such a case how do we estimate the outcome or the counter factual.
It’s a library released by Google for performing causal inferences in these kinds of cases. This library is available only in R, so I will run through the logic behind it and how it works. It takes care of such situations with the help of what is known as synthetic controls.
Take this graph of the Swiss Franc(CHF) to the Euro(EUR). For a long time, the Swiss Franc was pegged to the Euro and its value was stable. But after it was unpegged in 2015 it saw huge fluctuations. In this case, do we need a RCT so that we can determine what the Swiss Franc would have behaved like if it had not been unpegged?
No.
It is simple to say that it would have continued to be relatively stable with very few fluctuations. Now, this is what is known as a synthetic control. Now, this is possible here as it is a simple time series.
Take for instance this graph:
This is a more complex time series graph being influenced by multiple factors. Even state of the art techniques such as LSTM would find it difficult to predict it accurately. In this case, how do we predict the counter factual/synthetic control?
We can make use of other variables or covariates to help in estimating the synthetic control. The trick here is to find the covariates which are correlated to the desired metric but are not affected by the treatment.
In the above graph y is the desired metric and x1 and x2 below represent the two covariates.
In the pre-period, we train and test a ML model to predict the desired metric using the covariates.
In the post-period, we use the same model to predict the desired metric using the covariates. The predicted outcome is the synthetic control here. The actual outcome is then compared with the synthetic controls and the difference between the two is estimated to be the impact of the event on the outcome.
The above is how Google’s CausalImpact works in short. Now you can experiment with various models to see which works. This can be done by dividing the pre-period of your data into a training and validation section to identify the best model.
The Google package uses a Bayesian Structured Time Series model. For those of you who are unaware of how Bayesian Statistics works, I have written an article that you can access via this link. In short, Bayesian Statistics works by incorporating a prior belief on how something behaves like and over time updates the beliefs based on new data.
Most time series models, are just that, a time series model. They cannot take into account the external factors which affects the changes and fluctuations. The advantage of the Bayesian time series model is that it takes into account both the time series aspect as well as the external factors which influence the outcome. It incorporates the external factors or the covariates in this case by taking them as priors for the model. This way it can account for both the temporal evolution as well as the external factors while predicting.
Let's see a use of the CausalImpact package with some dummy data.
library(CausalImpact)set.seed(1)x1 <- 100 + arima.sim(model = list(ar = 0.999), n = 100)x2 <- 25 + arima.sim(model = list(ma = 0.246), n = 100)
I am importing the package and simulating two time series with the help of the arima.sim command. These will be the covariates.
y <- 1.2 * x1 + 0.8*x2 + rnorm(100)y[71:100] <- y[71:100] + 10
Here I am creating a new time series y which is a function of our two time series x1 and x2. For the last 30 observations, I am adding an extra to the value to simulate the effects of an event/treatment.
time.points <- seq.Date(as.Date("2020-01-01"), by = 1, length.out = 100)data <- zoo(cbind(y, x1, x2), time.points)matplot(data, type = "l")
Adding the date component to the series and plotting it.
pre.period <- as.Date(c("2020-01-01", "2020-03-10"))post.period <- as.Date(c("2020-03-11", "2020-04-09"))impact <- CausalImpact(data, pre.period, post.period)
I define the pre and post periods based on where I added the treatment effects and then pass on the data and information to the package.
plot(impact)
The first plot superimposes the original data and the predictions at every point along with the confidence interval.
The second plot displays the difference between the original and predicted data at every point along with the confidence interval.
The third plot displays the cumulative effect of the treatment by summing up the pointwise differences.
The ‘summary(impact)’ command can be used to give a tabular overview, while the ‘summary(impact, “report”) gives a written report of the findings.
From the above, we get a better idea of the effect of the treatment along with the confidence intervals. The summary even gives the probability of the change due to the treatment itself so that we know that it is not any spurious relation or change.
Causal Inference is a tricky problem that we data scientists rarely talk about. While this might be due to the fact that we generally don’t have to deal with this, still it is always good to know how to solve such a problem. It never hurts to have one more tool in our toolkit, one more weapon in our arsenal. Provided that the condition of finding covariates not influenced by the treatment is satisfied, this can indeed be a powerful technique to use in such scenarios.
I came across this technique when we were taking part in the NUS-NUHS-MIT Healthcare Datathon of 2019. Our problem statement revolved around estimating the impact of an event on outcomes, i.e. Causal Inference. By using this package we were able to successfully complete our task and also got the first place in the datathon. Below is the pic of our amazing team made up of the best teammates possible without which this would not have been possible:
You can connect with me on LinkedIn as well.