Regression Modelling and Variable Selection for the FTSE100
By: Caelan Dwyer
Tags: Regression, Algorithmic Trading, Finance, Stock Market, Data Science

Regression Modelling and Variable Selection for the FTSE100

Photo by Jamie Street on Unsplash

Supervised learning uses a trained algorithm to find patterns in a dataset. This article will focus on regression analysis, and how to build a suitable regression model to explain the variability in stock price of BAE systems using stock price data from other companies.

THE DATA — FTSE 100 Companies

The Financial Times Stock Exchange 100, also referred to as the FTSE 100, is an index composed of the 100 largest companies listed on the London Stock Exchange (LSE). The size of the company is determined by its Market Capitalization, which refers to the “total dollar market value of a company’s outstanding shares of stock” (Chen, 2020). This article examines the correlation between closing prices of companies within the FTSE 100 index.

The data gathered is from January 2016 to January 2019, and is held within a dataset with 1007 columns and 32 columns. The dataset looks at the closing share prices for BAE Systems (known as British Aerospace) and 27 other companies in the FTSE100 Index. The first four colomns were “Date”, “Year”, “Month”, and “Weekday”. The remaining 28 columns relate to the companies closing prices. Companies are displayed by their Stock Market Ticker (which is essentially an abbreviation relating to a publically traded company).

Image by Author

Libraries used within R:

library(tidyverse)

library(dplyr)

library(ggplot2)

library(GGally)

library(MASS)

Firstly I checked for missing values and then removed the first four columns to leave only the companies closing prices.

sum(is.na(dat))

dat <- lse %>% select(-Date, -Year,-Month,-Weekday)

Model Fitting & Interpretation

Before Multicollinearity

Using the corr() function I created a correlation matrix between all of the companies. The correlation coefficient is used in statistics to measure the strength of a relationship between two variables. The correlation coefficient is always between 1 and -1, inclusive. I have included a table looking at the correlation between each company and BAE systems.

Image by Author

I then created a more visually aesthetic plot called a correlogram to investigating the correlation between BA and the other 27 stocks within the Dataset (and also the correlation between covariates).

cor(dat)

ggcorr(dat,layout.exp = 1, hjust = 0.75, size = 3, label = TRUE, label_size = 2, label_round = 2, label_alpha = TRUE)

Image by Author

The 5 companies that have the strongest correlation (in absolute value) with BA, are St James Place, Centrica Plc, EasyJet, Rightmove plc, and the Vodafone Group plc. The absolute value of the sample correlation coefficient lies between 0 and 1.

Image by Author

Addressing Multicolleanrity

Looking at the correlogram it is clear that some companies are strongly correlated to each other, this suggest that there is multicollinearity. Multicollinearity occurs when independent variables within a multiple regression model are correlated. This is an issue within a regression model as the indendent variables should obviously be independent. When combinations of independent variables are strongly correlated the results of hypothesis tests, coefficient estimates and in turn interpretation of the model can be affected.

The following stocks have been removed due to a strong correlation between independent variables: SMT, NMC, SSE, MIN, RB, CCH, EXPN, SDR, MIN, SPX, VOD, GVC, RMV. During this process variables that were more related to the response were given retention priority. For example, CNA and VOD had a 0.87 correlation coefficient. However, CNA was more strongly correlated to BA, and contextually as an energy conglomerate it is more related in a business sense, therefore VOD was removed.

nomultdata <- dat %>%

select(-SMT, -NMC,-SSE, -MIN, -RB, -CCH, -EXPN, -SDR, -MIN, -SPX, -VOD, -GVC, -RMV)

nomultdata ggcorr(nomultdata,layout.exp = 1, hjust = 0.75, size = 3, label = TRUE, label_size = 2, label_round = 2, label_alpha = TRUE)

Image by Author

St James Place (STJ) is a British wealth management business, and in 2018 they invested 0.85% of their Net Portfolio Assets into BAE Systems, equating to 2,564,646 stocks (UK AND GENERAL PROGRESSIVE UNIT TRUST, 2018). This was a new holding and it could explain the relatively strong positive correlation to BAE System’s closing price.

Fitting the Regression Model — 5 most correlated

A multiple linear regression model was then fitted using the 5 companies with the strongest correlation to BA. This article will investigate the linear relationship between the predictors (5 companies in the table below) and the response (BA) using multiple linear regression. The OLS goal is to minimize the error (residual sum of squares) within the model.

Image by Author

The relationship between each of the five strongly correlated companies and BA appears to be linear, which will be useful when considering model transformations later on.

plot(BA~STJ, data=nomultdata)

plot(BA~CNA, data=nomultdata)

plot(BA~EZJ, data=nomultdata)

plot(BA~DLG, data=nomultdata)

plot(BA~AHT, data=nomultdata)

Image by Author

ANOVA

ANOVA is a statistical test that estimates how the dependent variable changes in accordance to independent variables. Specifically, it is used to analyze the difference between the means of more than two groups. Using table 4 we can carry out an F-test, using the qf() function to find the critical value, 2.223043, and pf() function to obtain the p-value of the test. The observed F (438.748) is very large in comparison with the critical value and our p-value is very small (2.692116e-249). Therefore, we must reject the null hypothesis H0, which indicates that there is a significant linear association between BA and at least one of the predictors (STJ, CNA, EZJ, DLG, AHT).

#creating the anova table manually

dfR <- nrow(nomultdata) - length(coef(model1))

dfM <- length(coef(model1))-1

dfT <- nrow(nomultdata) - 1

RSS <- sum(resid(model1)^2)

TSS <- sum((nomultdata$BA - mean(nomultdata$BA))^2)

MSS <- TSS - RSS

MSR <- RSS/dfR

MSM <- MSS/dfM

F <- MSM/MSR summary(model1)$r.sq #0.6866725 is the R squared value

summary(model1)$sigma #35.753 is the RSE value

pf(F, 5, 1001, lower.tail=FALSE)# 2.692116e-249

Image by Author

Assessing the model

model1 <- lm(BA~STJ+CNA+EZJ+DLG+AHT, data= nomultdata)

summary(model1)

Image by Author

The estimated regression line, obtained from the model summary, is:

BA = 562.721 + 55.514*STJ + 4.935*CNA — 16.822*EZJ — 16.051*DLG + 19.19*AHT

Looking at the model summary and the coefficients individually all of them have a p-value less than 0.05 and therefore we must reject the null hypothesis for each one, that there is a significant association. It is interesting that despite CNA being negatively correlated to BA, it is shown to have a positive additive effect on the predictor within the regression model. This could potentially have been caused by multicollinearity as CNA has a strong correlation to AHT, but it was not deemed strong enough to eliminate either variable. However, it is more likely that this phenomenon has been caused by an interaction between CNA and another covariate.

An R2 value near to 1 indicates that the model explains a large portion of the variance in the response variable. The model that uses the 5 closing prices of stocks most strongly correlated to BA has an R2 value of 0.6866725, and an RSE value of 35.753.

Regression Assumptions

The main assumptions that underpin the linear regression model are:

that the errors are normally distributed

the errors have constant variance

the errors are independent.

The normality of errors and constant variance assumptions will be investigated together using a Q-Q plot and a Residuals vs Fitted plot.

plot(model1) # non constant variance and errors are not normally distributed

Image by Author

Image by Author

The Quantile-Quantile plot is skewed left and indicates a departure from normality. Departures from normality are generally associated with the errors having a non-constant variance. Points in the random scatter plot (Residuals vs Fitted Values) are fanning out (smaller initial spread on the y axis), this is known as overdispersion and indicates a non-constant variance.

Attempting a transformation

A common variance stabilization transformation is log(Y). However, upon transforming the model the adjusted R squared value is smaller, 0.6839986. Therefore, it was decided model 1 (with no transformation) is better than model 2 (with log transformation). While the log transform is the most common variable transformation the box-cox method is used to pick the best. The plot below indicates that no transformation is the best strategy.

#box cox

library(MASS)

boxcox(model1, plotit= TRUE)

#specifying a range of lambda

boxcox(model1, plotit= TRUE, lambda = seq(0.5, 1.5, by = 0.1))

Image by Author

Selecting the best variables for the Regression Model

By introducing backwards stepwise regression this report intends to find the model with the optimal number of predictors. Starting with the full model the step function has been employed to eliminate underperforming predictors (the covariates that least reject the null hypothesis).

colnames(nomultdata) full.model <- lm(BA~STJ+PRU+AHT+CPG+CCL+RR+DLG+TUI+LLOY+EZJ+BATS+TSCO+CNA+RTO+ANTO, data=nomultdata)

summary(full.model) #using the BACKWARD step function we will eliminate under-performers

#reduced model

reduced.model <- step(full.model, direction = 'backward') #R squared value of reduced model

summary(reduced.model)$r.sq #0.8576528

Forward selection:

Forwards stepwise selection regression, starting with intercept only model and incrementally adding variables to improve the regression. The process stops when there can be no further improvement made through additional variables

# forward selection

# intercept model

min.model <- lm(BA~1,data=nomultdata)

fwd.model <- step(min.model, direction = 'forward', scope = (~STJ+PRU+AHT+CPG+CCL+RR+DLG+TUI+LLOY+EZJ+BATS+TSCO+CNA+RTO+ANTO))

summary(fwd.model) #R squared value of fwd model

summary(fwd.model)$r.sq #0.8576528

Step-wise selection:

min.model <- lm(BA~1,data=nomultdata)

step.model <- step(min.model, direction = 'both', scope = (~STJ+PRU+AHT+CPG+CCL+RR+DLG+TUI+LLOY+EZJ+BATS+TSCO+CNA+RTO+ANTO))

The same model was produced by both forward, backward, and step-wise selection. The resulting reduced model contains only significant predictors:

select_model = lm(BA ~ STJ + PRU + AHT + CPG + RR + DLG + LLOY + EZJ + BATS + TSCO + CNA + RTO + ANTO)

The reduced model has an R squared value of 0.8576528. This is larger than model 1 (5 stocks with the strongest correlation to BA).

The models produced by backwards and forwards stepwise selection regression are the same. The plots produced from the stepwise algorithms are shown below.

Image by Author

Image by Author

The QQ plot shows a phenomenon called thin tails, where more data is concentrated in the centre of the distribution and less in the tails. The thin tails correspond to the first quantiles occurring at larger than expected values, and the last at values smaller than expected (Kross, 2020). The random scatter plot displays a constant variance, as there is no obvious over or under dispersion. The plot below indicates that there are no data points out with Cooks distance, indicating that there is no data point significantly skewing the model results.

Image by Author

The independence assumption must be met otherwise it is likely that the model is underestimating the variability of our data. In turn, this would mean that the confidence intervals were too narrow and the hypothesis tests were invalid. Dependence can occur in sequentially collected data; therefore, this assumption must be investigated. A shapiro-wilks value of 0.99785 and a p-value of 0.2218 was calculated suggesting the sample does not deviate from normality. Looking at the scatterplots, produced using plot(select_model), there was no weaving or curvature visible. Thus, no strong evidence against independence exists.

Image by Author

Using maximum likelihood to estimate λ, the box cox plot shows that 0 is at the inverse of the parabola. Therefore, a natural log of the response could be taken (see appendix). The transformed model has an R squared value of 0.8614344, which is higher than select_model. A shapiro-wilks test indicates that this transformation actually causes a further deviation from normality the p-value is smaller (0.091). Therefore, the model should not be transformed this way.

Prediction and Validation

The RMSPE value to assess the fit of the model by comparing to the actual data.

Removing the natural log of BA reduced the RMSPE value from 515.98 to 71.13. This was a significant decrease and confirms that a natural log should not be taken of y (as first mentioned above). Introducing an interaction term ((all variables)²)) reduced the RMSPE value to 56.40.

I then looked at boxplots of all predictor variables to identify variables with outliers. It was identified that RR, LLOY, and RTO had a significant number of outliers. Therefore, I experimented removing each variable, and just removing RTO lowered the RMSPE value. Looking at the plots of independent variables vs BA it is clear that CNA has the least linear relationship with BA (potential quadratic), therefore I tried to transform CNA by interacting with other covariates. The model below produced a RMSPE value of 30.8. The predicted model follows the actual model quite closely. The final model also included an interaction term.

BAE.lm <- lm(BA ~ STJ + PRU + AHT + CPG + RR + DLG + LLOY + BATS + TSCO + CNA*EZJ + ANTO)

Image by Author

Full code available on GitHub.