CausalImpact and R: Analysing Time Series Interventions
By: Michael Grogan
Tags: Machine Learning, Rstats, Time Series, Data Science, R Programming

CausalImpact and R: Analysing Time Series Interventions

Analysing time series interventions in R

Source: Photo by Monoar_CGI_Artis from Pixabay

Disclaimer: This article is written on an “as is” basis and without warranty. It was written with the intention of providing an overview of data science concepts, and should not be interpreted as investment advice, or any other sort of professional advice.

Interventions (or external factors independent of a time series) can often influence said series.

For instance, a marketing campaign can influence the number of sales a company makes going forward. A change in government policy can significantly influence economic activity.

The main problem in analysing interventions is that it is not possible to analyse what would have happened had the intervention not occurred — at least not directly.

The CausalImpact library in R allows for analysis of intervention effects through using a separate series (one which is not affected by the intervention) as a covariate.

Our Example

In November 2017, the Bank of England decided to raise interest rates. When the central bank of a country decides to manipulate interest rates in this manner — the value of a currency can be affected — given that such a decision affects the rate that investors can yield through holding that particular currency.

Source: tradingeconomics.com

In this regard, we would like to examine if a rise in interest rates affected the value of the GBP/USD.

To do this, the EUR/USD currency is chosen as a covariate — this currency is hypothesised as not being affected by the intervention — as the ECB did not raise interest rates during this period.

Source: tradingeconomics.com

The relevant data is sourced from the FRED database using Quandl.

Here is a plot of the two currencies:

set.seed(1)

x1 <- eurusd$Value

y <- gbpusd$Value

data <- cbind(y, x1) dim(data)

head(data)

matplot(data, type = "l")

Source: RStudio

When working with financial time series, it is common to convert data into logarithmic format. While this makes sense when it comes to analysing stock prices that have significantly different price scales, the data is analysed “as is” in this example.

The reason is that currency data has a much smaller scale than most other asset prices, and using a logarithmic scale would bring the values so close together that any interventions that could be detected across the original series may not be detected when using a logarithmic scale.

Using this data, a pre-period and post-period is defined, with the data then plotted.

pre.period <- c(1, 230)

post.period <- c(231, 249) impact <- CausalImpact(data, pre.period, post.period)

plot(impact)

Source: RStudio

A summary of the impact is generated:

> summary(impact)

Posterior inference {CausalImpact} Average Cumulative

Actual 1.3 25.5

Prediction (s.d.) 1.3 (0.0099) 25.4 (0.1878)

95% CI [1.3, 1.4] [25.1, 25.8]



Absolute effect (s.d.) 0.0014 (0.0099) 0.0275 (0.1878)

95% CI [-0.018, 0.02] [-0.349, 0.38]



Relative effect (s.d.) 0.11% (0.74%) 0.11% (0.74%)

95% CI [-1.4%, 1.5%] [-1.4%, 1.5%] Posterior tail-area probability p: 0.44186

Posterior prob. of a causal effect: 56% For more details, type: summary(impact, "report")

We see that the predicted vs. actual values (average) are the same at 1.3, with only a slight difference across the cumulative values (25.5 vs. 25.4).

The posterior probability of a causal effect is 56%.

Let’s take a closer look at the generated summary report. Part of it states:

Although the intervention appears to have caused a positive effect, this effect is not statistically significant when considering the entire post-intervention period as a whole. Individual days or shorter stretches within the intervention period may of course still have had a significant effect, as indicated whenever the lower limit of the impact time series (lower plot) was above zero. The apparent effect could be the result of random fluctuations that are unrelated to the intervention. This is often the case when the intervention period is very long and includes much of the time when the effect has already worn off. It can also be the case when the intervention period is too short to distinguish the signal from the noise. Finally, failing to find a significant effect can happen when there are not enough control variables or when these variables do not correlate well with the response variable during the learning period. The probability of obtaining this effect by chance is p = 0.442. This means the effect may be spurious and would generally not be considered statistically significant.

Comparison with pycausalimpact — Python version

When the same example was analysed with pycausalimpact, the results came in slightly differently.

The model was defined as follows:

>>> ci = CausalImpact(data, pre_period, post_period)

>>> print(ci.summary())

>>> print(ci.summary(output='report'))

>>> ci.plot()

Here is the generated output:

Posterior Inference {Causal Impact}

Average Cumulative

Actual 1.34 25.46

Prediction (s.d.) 1.32 (0.0) 25.17 (0.08)

95% CI [1.32, 1.33] [25.01, 25.33] Absolute effect (s.d.) 0.02 (0.0) 0.29 (0.08)

95% CI [0.01, 0.02] [0.13, 0.45] Relative effect (s.d.) 1.15% (0.32%) 1.15% (0.32%)

95% CI [0.52%, 1.77%] [0.52%, 1.77%] Posterior tail-area probability p: 0.0

Posterior prob. of a causal effect: 100.0%

Unlike the model formulated in R, this model predicted a 100% posterior probability of a causal effect.

Source: Jupyter Notebook Output

The R and Python versions of this library differ in the way they analyse interventions. Specifically, R relies more on prior probabilities whereby previous knowledge is factored into the model assumptions, whereby Python relies more on analysis of the structural time series components in order to maximise the likelihood function.

That said, suppose the prior is now set to None, i.e. prior_level_sd=None.

ci = CausalImpact(data, pre_period, post_period, prior_level_sd=None)

print(ci.summary())

print(ci.summary(output='report'))

ci.plot()

Here are the results:

Posterior Inference {Causal Impact}

Average Cumulative

Actual 1.34 25.46

Prediction (s.d.) 1.35 (0.02) 25.57 (0.3)

95% CI [1.31, 1.38] [24.98, 26.15]



Absolute effect (s.d.) -0.01 (0.02) -0.11 (0.3)

95% CI [-0.04, 0.02] [-0.7, 0.47]



Relative effect (s.d.) -0.45% (1.17%) -0.45% (1.17%)

95% CI [-2.72%, 1.86%] [-2.72%, 1.86%]



Posterior tail-area probability p: 0.37

Posterior prob. of a causal effect: 63.34%

This model does predict a change in the rate from 1.34 to 1.35, but the model notes:

The probability of obtaining this effect by chance is p = 36.66%.

This means the effect may be spurious and would generally not be

considered statistically significant.

In this regard, the choice of prior can significantly affect the assumptions of the model and hence whether an intervention will ultimately be detected or not.

Conclusion

In this article, you have seen:

The use of CausalImpact in R to predict interventions

How the R and Python versions differ from each other

Role of prior assumptions in determining the detection of interventions

Intervention analysis and the generation of summary reports

Many thanks for reading, and any questions or feedback are greatly welcomed.

References