Macroeconomic & Financial Factors and Ordinary least Square Regression | by Sarit Maitra | Towards Data Science

Arbitrage Pricing Theory is represented as a multivariate regression model with across‐equations restrictions. It is a multi-factor asset pricing model that holds that an asset’s returns can be forecasted with the linear relationship of an asset’s expected returns and the macro-economic variables that capture systematic risk.
Our data here are monthly stock prices for the Brent Crude index, S&P500 index, US Treasury bill rate 3-month & 10 years (short-term & long-term), US treasury bill rate (credit risk), the consumer price index, industrial production index.
We would like to examine here, whether Brent Crude Oil prices can be explained with the support of unexpected changes in a set of macroeconomic and financial variables. The best practice while working with stock prices is to use adjusted values, as they account for possible corporate actions, such as stock splits.
combine = pd.concat([h['SP500'], g, f['DGS10'], e['DTB3'], d['TEDRATE'], b['TOTALSL'],a['CPALTT01USM657N'], i['Brent spot price']], axis=1)combine.set_index('DATE', inplace=True)combine.index = pd.to_datetime(combine.index)combine.dropna(inplace=True)
Let us generate a set of changes or differences for each of the variables. The unexpected value of a variable can be defined as the difference between the actual value of the variable and its expected value.
Tao, R., & Brooks, C have presented a nice article on arbitrage pricing analysis, the reference of which I have added in the end of this article.
We will use logarithmic returns here as they aggregate over time; the log return for a given month is the sum of the log returns of the days within that month.
here, rt is the return, Pt is the price of an asset in time t.
data = DataFrame({'ted_spread' : combine['TEDRATE'] - combine['TEDRATE'].shift(1), 'consumer_credit' : combine['TOTALSL'] - combine['TOTALSL'].shift(1),'indprod' : combine['INDPRO'] - combine['INDPRO'].shift(1),'cr_crude' : np.log(combine['Brent spot price'].diff().dropna()),# (cr_crude) continuously compounded returns for crude oil'cr_sp' : np.log(combine['SP500']).diff().dropna()# (cr_sp) continuously compounded returns for S&P500'inflation' : np.log(combine['CPALTT01USM657N']).diff().dropna(),# (inflation) continuously compounded returns for consumer price index'dgs_dtb' : combine['DGS10'] - combine['DTB3'],'inflation_diff' : np.log(combine['CPALTT01USM657N']).diff().dropna() - np.log(combine['CPALTT01USM657N']).diff().dropna().shift(1),'dtb' : combine['DTB3']/12,'dgs_dtb_shift' : (combine['DGS10'] - combine['DTB3']) - (combine['DGS10'] - combine['DTB3']).shift(1),'crude_dtb' : np.log(combine['Brent spot price']).diff().dropna() - combine['DTB3']/12,# (crude_dtb3) to transform the returns, (Crude continuously compounded returns) - (3-month US Treasury bill rate)# = risk free rate'sp_dtb' : np.log(combine['SP500']).diff().dropna() - combine['DTB3']/12})# (sp_dtb3) (SP continuously compounded returns) - (3-month US Treasury bill rate)
OLS regression serves as a good prototype for ML algorithms in general. It’s a supervised algorithm that has a training procedure and a deployment procedure. It can be over-fit in some circumstances. It shares these features with other more modern function approximation algorithms. However, limitation with OLS is that, there’s no means to stop the training when it over fits.
We will use Ordinary least squares (OLS) regression to estimate the relationship between independent variables and dependent variable (crude oil here) by minimizing the sum of the squares in the difference between the observed and predicted values of the dependent variable configured as a straight line.
data = data[~data.isin([np.nan, np.inf, -np.inf]).any(1)]X = data [['cr_sp', 'indprod', 'consumer_credit', 'inflation_diff', 'ted_spread', 'dgs_dtb_shift']]y = data['cr_crude']model = sm.OLS(y, sm.add_constant(X))model_fit = model.fit()model_fit.summary()
The regression F-statistic value is 10.71. The p-value < 0.05 for indprod & dgs_dtb_shift means, null hypothesis can be rejected for these variables. p-values for rest of the variables are > 0.05. Let us test the null hypothesis that the parameters on these four variables are jointly zero using F-test.
The F-test statistic value is 2.764; p-value 0.028 with there are four restrictions and 239 usable observations. This suggests that the null hypothesis cannot be rejected.
Diagnostic plots are drawn using residuals of the model. Let us visualize these plots to see how our regression model is fitting the data and if any of the basic assumptions of an OLS model are being violated. The diagnostic plots show residuals in four different ways.
# model valuesy_fitted = model_fit.fittedvalues# model residualsmodel_residuals = model_fit.resid# normalized residualsmodel_norm_residuals = model_fit.get_influence().resid_studentized_internal# absolute squared normalized residualsmodel_norm_residuals_abs_sqrt =np.sqrt(np.abs(model_norm_residuals))# absolute residualsmodel_abs_resid = np.abs(model_residuals)# leverage, from statsmodels internalsmodel_leverage = model_fit.get_influence().hat_matrix_diag# cook's distance, from statsmodels internalsmodel_cooks = model_fit.get_influence().cooks_distance[0]
dataframe = pd.concat([X, y], axis=1)plot_lm_1 = plt.figure()plot_lm_1.axes[0] = sns.residplot(y_fitted, dataframe.columns[-1], data=dataframe, lowess=True, scatter_kws={'alpha': 0.5}, line_kws={'color': 'red', 'lw': 1, 'alpha': 0.8})plot_lm_1.axes[0].set_title('Residuals vs Fitted')plot_lm_1.axes[0].set_xlabel('Fitted values')plot_lm_1.axes[0].set_ylabel('Residuals');
Though we do not see any distinctive pattern from the above plot, it seems like, some of the non-linear features of the model are not captured well which means we are under-fitting the model. However, we will check QQ plot to determine the normality assumption.
This plot shows if the residuals are normally distributed. A good normal QQ plot has all of the residuals lying on or close to the red line.
QQ = ProbPlot(model_norm_residuals)plot_lm_2 = QQ.qqplot(line='45', alpha=0.5, color='#4C72B0', lw=1)plot_lm_2.axes[0].set_title('Normal Q-Q')plot_lm_2.axes[0].set_xlabel('Theoretical Quantiles')plot_lm_2.axes[0].set_ylabel('Standardized Residuals');# annotationsabs_norm_resid = np.flip(np.argsort(np.abs(model_norm_residuals)), 0)abs_norm_resid_top_3 = abs_norm_resid[:3]for r, i in enumerate(abs_norm_resid_top_3):plot_lm_2.axes[0].annotate(i,xy=(np.flip(QQ.theoretical_quantiles, 0)[r],model_norm_residuals[i]));
Here, most of the data points fall along the red line; Of course they wouldn’t be a perfect straight line and it looks like normality assumption is met.
Let us look at the histogram to check a normal distribution; here we expect a histogram of the residuals to be bell-shaped (with no outliers).
The histogram confirms the distribution of the residuals roughly resembles a bell-shape; although, we also find that there are some large negative outliers which might lead to a considerable negative skewness.
SkewnKurtosis measures extreme values in either tail. The kurtosis for a standard normal distribution is 3. Kurtosis with values 6.11 means residuals exceeding the tails of the normal distribution. However, in general, these inferences could be wrong; our sample is probably large enough that we need be less concerned than we would with a smaller sample.
This plot shows if residuals are spread equally along the ranges of predictors. We can check the assumption of equal variance which is homoscedasticity.
plot_lm_3 = plt.figure()plt.scatter(model_fitted_y, model_norm_residuals_abs_sqrt, alpha=0.5);sns.regplot(model_fitted_y, model_norm_residuals_abs_sqrt, scatter=False, ci=False,lowess=True, line_kws={'color': 'red', 'lw': 1, 'alpha': 0.8});plot_lm_3.axes[0].set_title('Scale-Location')plot_lm_3.axes[0].set_xlabel('Fitted values')plot_lm_3.axes[0].set_ylabel('$\sqrt{|Standardized Residuals|}$');# annotationsabs_sq_norm_resid = np.flip(np.argsort(model_norm_residuals_abs_sqrt), 0)abs_sq_norm_resid_top_3 = abs_sq_norm_resid[:3]for i in abs_norm_resid_top_3:plot_lm_3.axes[0].annotate(i,xy=(model_fitted_y[i],model_norm_residuals_abs_sqrt[i]));
The residuals appear randomly spread which is an indication of homoscedasticity. Here, the error term is the same across all values of the independent variables. This assumption of homoscedasticity is central to linear regression models.
plt.figure(figsize = (20,5))plt.plot(model_fit.resid)plt.xlabel('Date')plt.ylabel('Residuals')plt.grid(True)
By examining the patterns in the residuals over time we can also confirm homoscedastic nature of residuals.
From the test statistics and p-values, both tests lead to the conclusion that residuals are homoscedastic nature for our model with the p-values both being 0.0001 and 8.21, respectively.
This plot helps us to find outliers if any. Even though data have extreme values, they might not be influential to determine a regression line. That means, the results wouldn’t be much different if we either include or exclude them from analysis. We check for outlying values at the upper right corner or at the lower right corner. Those spots are the places where cases can be influential against a regression line.
plot_lm_4 = plt.figure();plt.scatter(model_leverage, model_norm_residuals, alpha=0.5);sns.regplot(model_leverage, model_norm_residuals,scatter=False, ci=False,lowess=True, line_kws={'color': 'red', 'lw': 1, 'alpha': 0.8});plot_lm_4.axes[0].set_xlim(0, max(model_leverage)+0.01)plot_lm_4.axes[0].set_ylim(-3, 5)plot_lm_4.axes[0].set_title('Residuals vs Leverage')plot_lm_4.axes[0].set_xlabel('Leverage')plot_lm_4.axes[0].set_ylabel('Standardized Residuals');# annotationsleverage_top_3 = np.flip(np.argsort(model_cooks), 0)[:3]for i in leverage_top_3:plot_lm_4.axes[0].annotate(i,xy=(model_leverage[i], model_norm_residuals[i]));
We only need to find leverage points that have a distance greater than 0.5. In this plot, all cases are well inside of the Cook’s distance. The plot identified the influential observation as #245. If we exclude the 245th case from the analysis, the slope coefficient and R2 values likely to improve considerably.
Through step-by-step we will iteratively construct regression model which involves automatic selection of independent variables. Python’s statsmodels doesn’t have a built-in method for choosing a linear model by forward selection. I have taken this code from here. I have used p-values threshold <0.20 to filter.
formula = 'cr_crude ~ cr_sp + indprod + consumer_credit + inflation_diff + ted_spread + dgs_dtb_shift'results = smf.ols(formula, data).fit()def forward_selected(data, endog, exg):remaining = set(data.columns)remaining = [e for e in remaining if (e not in endog)&(e not in exg)]exg = [exg]scores_with_candidates = []for candidate in remaining:formula = '{} ~ {}'.format(endog,' + '.join(exg + [candidate]))score = model_fit.pvalues[2]scores_with_candidates.append((score, candidate))scores_with_candidates.sort()for pval,candidate in scores_with_candidates:if pval < 0.2:exg.append(candidate)formula = '{} ~ {}'.format(endog, ' + '.join(exg))model = smf.ols(formula, data).fit()return modelmodel = forward_selected(data,'cr_crude','cr_sp')print(model.model.formula)cr_crude ~ cr_sp + consumer_credit
As can be seen, the regression formula crude ~ cr_sp + consumer_credit, suggesting that the S&P500 and CPI are the only determinant factors for crude oil price.
A best way to test for errors in this model is to not rely on the model’s F-statistic, significance, or R2, but instead assess the model against test-set. Accuracy is then often measured as the actual standard error, MAPE (Mean absolute percentage error), or mean error between the predicted value and the actual value in the test-data.
From the graph, we could sense the presence of some outliers; especially during 2008 is quite visible. We will have a better idea if we look at the IQR box plot of all variables.
Here, we can clearly see some outliers. Let us identify the exact dates of the outliers.
We could see that, 2 biggest outliers are on Oct & Nov of 2008. Let us create 5 extra variables with the outliers and add those with the existing data frame to run the regression again.
data['nov_2008'] = np.where(data.index == '2008-11-01', 1, 0)data['oct_2008'] = np.where(data.index == '2008-10-01', 1, 0)data['dec_2014'] = np.where(data.index == '2014-12-01', 1, 0)data['jan_2015'] = np.where(data.index == '2015-01-01', 1, 0)data['apr_2000'] = np.where(data.index == '2000-04-01', 1, 0)formula = 'cr_crude ~ cr_sp + indprod + consumer_credit + inflation_diff + ted_spread + dgs_dtb_shift +nov_2008 + oct_2008 + dec_2014 + jan_2015 + apr_2000'model = smf.ols(formula, data).fit()model.summary()
Here, we see that, all the added variables are highly significant p-values <0.05). If we compare the results, we can see that the coefficient estimates on the remaining variables remain unchanged in this instance. The significant parameters remains the same as indpro, dgs_dtb_shift parameters; Adj. R2 value has improved from 19.2% to 34.6%.
col = data[['indprod', 'consumer_credit', 'inflation_diff', 'ted_spread', 'dgs_dtb_shift']]plt.figure(figsize = (10,6))pearsoncorr = col.corr(method='pearson')sns.heatmap(pearsoncorr, xticklabels=pearsoncorr.columns,yticklabels=pearsoncorr.columns,cmap='RdBu_r',annot=True,linewidth=0.5)
Here, the largest observed correlations are 0.37 between the consumer_credit and inddprod variables, and -0.017 between the inddprod and inflation_diff. Both are quite small in real value and they can reasonably be ignored.
We have used different macro-economic factors and financial variables to estimate whether monthly return on Brent Crude oil can be explained considering unexpected changes in those variables. The regression results show that there are quite small differences among crude oil return against independent variables that we had chosen. Our statistical analysis showed that, only he changes on short-term & long-term US treasury bill yields index and the Industrial Production index are significant among the variables chosen for our analysis to determine monthly return on Brent Crude Price. However, the regression results indicate that, there is no significant pricing relation between the stock return and the tested macroeconomic and financial variables.
Connect me here.
Reference:
Gauss, Carl Friedrich. [1809] 2004. Theory of Motion of the Heavenly Bodies Moving About the Sun in Conic Sections: A Translation of Theoria Motus. Mineola, NY: Dover.Tao, R., & Brooks, C. (2019). Python Guide to Accompany Introductory Econometrics for Finance. Available at SSRN 3475303.
Gauss, Carl Friedrich. [1809] 2004. Theory of Motion of the Heavenly Bodies Moving About the Sun in Conic Sections: A Translation of Theoria Motus. Mineola, NY: Dover.
Tao, R., & Brooks, C. (2019). Python Guide to Accompany Introductory Econometrics for Finance. Available at SSRN 3475303.