Quantify, Understand, Model and Predict Financial Time-Series
By: Sarit Maitra
Tags: Time Series Analysis, Momentum Trading, Brownian Motion, Econometrics

Hurst Exponent to determine momentum of time-series

Quantify, Understand, Model and Predict Financial Time-Series

Image by author

FORECASTING stock price accurately is a formidable task. Irregular temporal behavior is omnipresent in the stock market prices due to the inherent complex behavior. Here, I will show a predictive modeling framework for forecasting the future returns of financial markets with a 3-stage approach.

Fractal modeling and recurrence analysis, and test the efficient market hypothesis to comprehend the temporal behavior in order to investigate autoregressive properties. Granger causality tests in a VAR environment to explore the causal interaction structures and identify the explanatory variables for predictive analytics. ML and ANN algorithms to learn the inherent patterns and predicting future movements.

I will use the precious metals (Gold & Silver) stock prices, Energy stock (Crude Oil) prices and S&P500 index since 2000 -2019. The objective is to estimate the future returns and the nexus of these products with S&P index.

Data pre-processing & Descriptive statistics:

After doing all the necessary data purification, filling the missing values with previous ones, renaming the column, indexing date etc. following is the output where we can do further analysis and investigation. Data purification takes a lot of time and effort. So, I will skip the explanations on data handling procedures and rather focus on the objective here.

if __name__ == "__main__":

df = pd.read_excel('brownian data.xlsx', index_col=0)

# removing unnecessary columns

df = df.drop(['Date', 'Year', 'NaturalGasPrice'], axis=1)

# converting 'DatesAvailable' to datetime

df['DatesAvailable']= pd.to_datetime(df['DatesAvailable'])

# setting 'DatesAvailable' as index column

df.set_index('DatesAvailable', inplace=True)

df1 = pd.read_excel('S&P Futures.xlsx', parse_dates = True,

index_col=0)

DF = df1.loc['20190729':'20000104']

data = [DF,df] # combining dataframes

result = pd.concat(data, axis=1)

# renaming column

result = result.rename(columns = {"Price": "SPPrice"})

DATA = result.fillna(method ='pad')

# filling the missing values with previous ones

DATA.index.name = 'Date'

print(DATA.head())

DATA.plot(subplots=True, layout = (2,2),

figsize = (15,6), sharex = False, grid =True)

plt.tight_layout()

plt.show()

Daily closing prices are taken for all the series. The plots exhibit a clear trend for S&P Index; for commodities quite random in nature. Visually it can be assumed that, none of the series are stationary and commodities have a Brownian motion movement. The results are in table format for the ease of understanding.

Skewness and kurtosis values confirm that none of the series follow normal distribution. The percentiles along with the standard deviation suggest a large spread for Index and Gold Price. The large spread for Gold in this series will likely make accurate predictions difficult if it is caused by random fluctuation.

Mann-Whitney test

The null hypothesis of Mann-Whitney U test is that there is no difference between the distributions of the data samples.

Unit root test Original series

The test statistics confirm that none of the series follow normal distribution. This is also an indication of non-parametric and nonstationary series and justifies the deployment of advanced ML and DNN algorithms for predictive modeling exercise. However, we won’t get a stable result if the time series follows a Brownian motion. So, it is important to test the RWH (Random walk hypothesis).

Unit root test 1st order difference

The outcome revealed that the series is first order stationary. Hence, I will consider the return series for econometric analysis to assess the direction of causation.

Non-linear dynamics:

Here, we will perform fractal modeling and recurrence quantification analysis (RQA) to check RWH and gain deeper insights about temporal evolutionary patterns. Let’s look into FD (fractal dimension), R/S (rescaled range), H(Hurst exponent) and RQA (recurrence quantification analysis).

Let us understand Hurst Exponent. The Hurst exponent is used as a measure of long-term memory of timeseries. The goal of the Hurst Exponent is to provide us with a scalar value that will help us to identify whether a series is mean reverting, random walk or trending. It is a statistical inference. In particular:

H < 0:5 — Anti-persistent time-series which roughly translates to mean reverting.

H = 0:5 — The time series is random walk and prediction of future based on past data is not possible.

H > 0:5 — Persistent time-series which roughly translates to trending.

FD = ln(N)/ln(1/d), N = number of circles, d = diameter. This equation shows how the number of circles related to the diameter of the circle. The value of FD lies between 1 and 2 for a time series. The FD of Brownian motion is 1.5. If 1.5 < FD < 2, then a time series is an anti-persistent process, and if 1 < FD < 1.5, then the series is a long memory process (persistent). H is related to FD (FD =2-H) and a characteristic parameter of long-range dependence. In case of H, the value of 0.5 signifies no long-term memory, < 0.5 means anti-persistence and > 0.5 means that the process is correlated across time and is persistent. R/S is the central tool of FD and calculated from dividing the range of its mean adjusted cumulative deviate series by the standard deviation of the time series itself (1). It is the measure characterizing the divergence of time series defined as the range of the mean-centered values for a given duration (T) divided by the standard deviation for that duration (2).

Recurrence Quantification Analysis (RQA) can deal with non-stationarity in the series and contribute to the understanding of the complex dynamics hidden in financial markets.

Now, let’s use the quantification technique by computing the measures like REC, DET, TT, and LAM.

Compute H:

if __name__ == "__main__":

index = DATA[['SPIndex']]

lag1, lag2 = 2,20

lags = range(lag1, lag2)

tau = [sqrt(std(subtract(index[lag:], index[:-lag]))) for lag in lags]

m = polyfit(log(lags), log(tau), 1)

hurst = m[0]*2



gold = DATA[['GoldPrice']]

lag1, lag2 = 2,20

lags = range(lag1, lag2)

tau = [sqrt(std(subtract(gold[lag:], gold[:-lag]))) for lag in lags]

m = polyfit(log(lags), log(tau), 1)

hurst = m[0]*2 crude = DATA[['CrudeOilPrice']]

lag1, lag2 = 2,20

lags = range(lag1, lag2)

tau = [sqrt(std(subtract(crude[lag:], crude[:-lag]))) for lag in lags]

m = polyfit(log(lags), log(tau), 1)

hurst = m[0]*2 silver = DATA[['SilverPrice']]

lag1, lag2 = 2,20

lags = range(lag1, lag2)

tau = [sqrt(std(subtract(silver[lag:], silver[:-lag]))) for lag in lags]

m = polyfit(log(lags), log(tau), 1)

hurst = m[0]*2 print('*'*60)

print( 'hurst (Index), 2-20 lags = ',hurst[0])

print('*'*60)

print( 'hurst (Crude), 2-20 lags = ',hurst[0])

print('*'*60)

print( 'hurst (Gold), 2-20 lags = ',hurst[0])

print('*'*60)

print( 'hurst (Silver), 2-20 lags = ',hurst[0])

np.random.seed(42)

random_changes = 1. + np.random.randn(5019) / 1000.

DATA.index = np.cumprod(random_changes)

H, c, result = compute_Hc(DATA.SPIndex, kind='price',

simplified=True) plt.rcParams['figure.figsize'] = 10, 5

f, ax = plt.subplots()

_ = ax.plot(result[0], c*result[0]**H)

_ = ax.scatter(result[0], result[1])

_ = ax.set_xscale('log')

_ = ax.set_yscale('log')

_ = ax.set_xlabel('log(time interval)')

_ = ax.set_ylabel('log(R/S ratio)')

print("H={:.3f}, c={:.3f}".format(H,c))

The Hurst exponent ‘H’ is the slope of the plot of each range’s log(R/S) versus each range’s log(size). Here log(R/S) is the dependent or the y variable and log(size) is the independent or the x variable. This value indicates that our data is a persistent one. However, we are working on a small data set and it cannot be concluded from the output that, H values are significantly higher, especially the commodities values (0.585) but the given time series has some degree of predictability. RQA will help us to understand the degree of predictability.

Compute RQA:

time_series = TimeSeries(DATA.SPIndex, embedding_dimension=2,

time_delay=2)

settings = Settings(time_series,

analysis_type=Classic,

neighbourhood=FixedRadius(0.65),

similarity_measure=EuclideanMetric,

theiler_corrector=1) computation = RQAComputation.create(settings,verbose=True)

result = computation.run()

result.min_diagonal_line_length = 2

result.min_vertical_line_length = 2

result.min_white_vertical_line_lelngth = 2

print(result)

Here, we see that, the RR (recurrence rate) values of all the time series are not on a higher side indicating a lower degree of periodicity. DET (S&P, Crude & Silver) & LAM values Crude, Gold & Silver) are higher supporting the deterministic structure. However, this also confirms the presence of higher order deterministic chaos.

Econometric approach:

A series of tests procedures have been employed here to explore the causal interaction structures among the variables and identify the explanatory variables for predictive analytics.

Pearson correlations display significant correlations between S&P & Gold, Crude & Gold, Crude & Silver and Gold & Silver. Instantaneous Phase Synchrony (IPA) and Granger Causality tests were performed.

Empirical studies suggest that, even a strong correlation between two variables is not a guarantor of causality. It does not provide information about directionality between the two signals such as which signal leads and which follows.

Now, I will use Granger causality test to inspect the causal interrelationship for identifying predictors. VAR was considered a means of conducting Granger causality tests. In the VAR model, each variable is modelled as a linear combination of past values of itself and the past values of other variables in the system. We have 4 time series that influence each other, so, we will have a system of 4 equations.

Y1, t = α1 + β11, 1Y1, t-1 + β12, 1Y2, t-1 + β13, 1Y3, t-1 + β14, 1Y4, t-1 + ε1, t Y2, t = α2 + β21, 1Y1, t-1 + β22, 1Y2, t-1 + β23, 1Y3, t-1 + β24, 1Y4, t-1 + ε2, t Y3, t = α3 + β31, 1Y1, t-1 + β32, 1Y2, t-1 + β33, 1Y3, t-1 + β34, 1Y4, t-1 + ε3, t Y4, t = α4 + β41, 1Y1, t-1 + β42, 1Y2, t-1 + β43, 1Y3, t-1 + β44, 1Y4, t-1 + ε4, t

Here, Y{1,t-1, Y{2,t-1}, Y{3,t-1}, Y{4,t-1} are the first lag of time series Y1,Y2, Y3, Y4 respectively. The above equations are referred to as a VAR (1) model, because, each equation is of order 1, that is, it contains up to one lag of each of the predictors (Y1, Y2, Y3 and Y4). Since the Y terms in the equations are interrelated, the Y’s are considered as endogenous variables, rather than as exogenous predictors. To thwart the issue of structural instability, I have used VAR framework choosing the lag length according to AIC.

# make a VAR model

model = VAR(DATA_diff)

model.select_order(12)

x = model.select_order(maxlags=12)

x.summary()

Lag selection (VAR)

The lowest value of AIC obtained at lag 4. So, causality analysis is carried out on this basis.

A = model.fit(maxlags=4, ic=’aic’) # pass a maximum number of lags and the order criterion to use for order selection R1=A.test_causality(‘Index’, [‘CrudeOilPrice’, ‘GoldPrice’, ‘SilverPrice’], kind=’f’) R2=A.test_causality(‘CrudeOilPrice’, [‘Index’, ‘GoldPrice’, ‘SilverPrice’], kind=’f’) R3=A.test_causality(‘GoldPrice’, [‘Index’, ‘CrudeOilPrice’, ‘SilverPrice’], kind=’f’) R4=A.test_causality(‘SilverPrice’, [‘Index’, ‘GoldPrice’, ‘CrudeOilPrice’], kind=’f’) R5=A.test_causality(‘Index’, [‘CrudeOilPrice’], kind=’f’)

Granger Casuality test result

We can see here from above table that, S&P Index does not cause the given other stocks prices in the series but other stocks prices have significant impact on S&P Index. Therefore, the causal structure is unidirectional here.

granger_test_result = grangercausalitytests(DATA_diff[[‘Index’, ‘CrudeOilPrice’]].values,maxlag=4) granger_test_result = grangercausalitytests(DATA_diff[[‘Index’, ‘GoldPrice’]].values,maxlag=4) granger_test_result = grangercausalitytests(DATA_diff[[‘Index’, ‘SilverPrice’]].values,maxlag=4) granger_test_result = grangercausalitytests(DATA_diff[[‘CrudeOilPrice’, ‘SilverPrice’]].values,maxlag=4) granger_test_result = grangercausalitytests(DATA_diff[[‘CrudeOilPrice’, ‘GoldPrice’]].values,maxlag=4) granger_test_result = grangercausalitytests(DATA_diff[[‘GoldPrice’, ‘SilverPrice’]].values,maxlag=4)

Next, the IR (impulse response) is estimated for assessing the impacts of shock from one asset on another.

IR plot has displays the expected level of the shock in a given period, the dotted lines represent 95% Confidence Interval (a low estimate and a high estimate). Further the % of variance in returns of all the four series are largely explained by themselves in the short run. In the short run, shocks of

their own movement tend to play a significant role in volatility. Series that are found to possess impact on the others by Granger causality, have marginal impact on variance as well. Hence, the overall findings of causation analysis are validated through the outcomes of variance decomposition. Results of causal interactions help in comprehending the structure of interrelationships. To further justify, forecast error variance decomposition (FEVD) is performed. The basic idea is to decompose the variance-covariance matrix so that Σ=PP−1, where P is a lower triangular matrix with positive diagonal elements, which is obtained by a Chelsi decomposition.

Forecast error variance decomposition

FEVD indicates the amount of information each variable contributes to the other variables in the auto-regression. It determines how much of the forecast error variance of each of the variables can be explained by exogenous shocks to the other variables. The percentages of variance in returns of all the three stock indexes are largely explained by themselves in the short run.

However, Granger causality do not necessarily reflect true causality. Therefore, for rigor, statistically verified the existence of the synchronization between the assets using IPA (Instantaneous phase synchrony). Here IPA is used for visual pattern recognition. It measures by incrementally shifting one time series vector and repeatedly calculating the correlation between two signals. The peak correlation at the center indicates the two-time series are most synchronized at that time. The transformation to the frequency domain achieved by Hilbert transformation after bandpass filtering.

The preprocessing stage, prior to performing any phase synchronization require de-trending the ﬁnancial data and removing the high-frequency noise components. The ﬁltering stage uses the SST as a bandpass ﬁlter (SST is a block algorithm, so the data was ﬁrst windowed), applied to only the low frequency oscillations of the de-trended signal. An important criterion for the band-pass ﬁltering is to ensure that the resulting signal is as narrowband as possible.

Pearson r is a measure of global synchrony which reduces the relationship between two signals to a single value. However, we have used instantaneous phase synchrony measure here to compute moment-to-moment synchrony between two signals without arbitrarily deciding the window size as done in rolling window correlations.

S&P Index & Crude Oil

S&P Index and Gold

S&P Index and Silver

Crude Oil & Gold

Crude Oil & Silver

Gold & Silver

Each plot consists of filtered time series (top), angle of each signal at each moment in time (middle row), and IPS measure (bottom). When two signals line up in phase their angular difference becomes zero. The angles are calculated through the Hilbert transform of the signal. Here, I have simulated a signal with phase synchrony. Both signals y1 and y2 are at 30Hz (30 days) for each of the plots. I have added random noise to the data and see how it changes the results. The fluctuating phase synchrony graph indicates that IPS is sensitive to noise and highlights the importance of filtering and choosing a frequency band for analysis. Simple moving-average and bandpass filters were employed to remove this noise from the signals, before moving on to the actual processing of waveforms. However, the IPA yields a global measure. Hence, during decomposing the signal into multi-components is a key criterion is to ensure the associated frequency is locally valid. The presence and duration of synchronization reflect dynamics of financial market.

Predictive model

Selection of explanatory constructs is pivotal for predictive modelling. Some widely used technical indicators are SMA, MACD and CCI are used for trend, STD and BB were used for volatility, RSI and WR were used for momentum. Therefore, the model tested with multivariate framework using Crude oil, Gold & Silver as input variable that significantly impact S&P index and a set of technical indicators as listed below.

The original dataset was randomly split into 85% training and 15% validation data. An algo search was performed on kNN, DT, RF, GBM, SVN and DNN to select a robust model to train on the given data set.

The objective is to get the lowest error and by that logic, it can be seen that GBM/XGB displays lowest error rate on both training and testing data. This justifies the use of GBM algo to develop predictive model for the given time series. DNN is known for their ability to handle continuous and high-volume data. Therefore, DNN was also chosen to develop a hybrid predictive model which can deal with high velocity big data.

I have performed heavy duty performance tuning with 5-fold CV technique to obtain the best of the two algos (XGB & DNN). Finally used stacked ensemble technique where combination of the two for superior predictive performance.

The average MSE, RMSE, and MAE values are found to be small for Crude Oil and Silver indicating effective predictive power of the model for all. The result also supports the fact that historically Gold has tended to be resilient during stock market crashes and the two are negatively correlated. Here also it shows no exception with high MSE value. Let’s see how our features are prioritized.

Feature importance-XGB

Feature importance-DNN

The features are plotted as per their relative strength and according to their order of importance. This implies that almost all the features selected, have some degree of importance to the model building.

Conclusion

Here, I have explained the key characteristics of selected stocks and S&P and subsequently delves into their causal interrelationship and predictive analysis. Some key technical indicators are explained here. This forecasting structure is a combination of econometric model and machine learning algorithms. Though I have tested with ML algos, however, technical indicators can also be used for a successful trading strategy. This has to be followed by a roll forward validation on validation set.

Connect me here.

Reference:

(1) Mandelbrot, B. B., & Wallis, J. R. (1969). Computer experiments with fractional Gaussian noises: Part 2, rescaled ranges and spectra. Water resources research, 5(1), 242–259.

(2) Hurst, H. E., Black, R. P., & Simaika, Y. M. (1965). Long-term storage: an experimental study Constable. London UK.