Applying Two Ideas from Advances in Financial Machine Learning
By: Neo Yi Peng
Tags: Statistics, Quantitative Strategies, Data Science, Forecasting, Machine Learning

When I’m not tweaking my soccer betting models, I’m dabbling in finance. Recently, I’ve been fascinated by this book, Advances in Financial Machine Learning (AFML)— the author combines academic rigour with practical execution. It’s not an easy read — I’ve had to re-read chapters and references a few times before I really got what he was saying, but it’s well worth it.

At the same time, I stumbled upon this online trading challenge by WorldQuant. At first, I was put off because you had to code in their proprietary language and not Python, but as I played around with the platform I found that it was quite easy to pick up and allowed me to focus on ideas generation and execution without worrying about Python packages and environment setups.

I will go through two feature extraction ideas from AFML, where he talks about how to extract features using Structural Breaks and Entropy.

Update: My team made it to the national finals in Singapore! Find a section of the slides that covers the strategies here.

1. Structural Breaks

The first thing people usually think of when applying Machine Learning to finance is to predict the future, usually the future price of an asset. But we can also use it to predict situations where certain strategies will work better. Structural breaks are one such example.

A structural break represents a regime change, implying the new price observed does not belong to the original distribution of the price series so far. A break could indicate a bubble formation. During breaks, there are opportunities as it is unexpected — market players scramble to adjust to the new norm, hence certain strategies may work better.

I calculated structural breaks via the ‘CUSUM’ test, i.e. measuring cumulative noises that cross a threshold, implying a break. Chu, Stinchcombe, and White[1] describe a simple way of calculating the departure of log-prices; I implemented one such iteration using the WorldQuant platform below.

snt: Test Statistic; crit: Critical Value

2. Approximating Entropy using Kurtosis

Entropy was first used in thermodynamics as a measure of irreversible heat loss [2]. Claude Shannon, the father of information theory, applied the concept to information entropy [3], a measure of how much value is contained in a message, based on how probable it is — the lower the probability, the higher the information content, and vice versa.

So how does this relate to the markets? Some have linked market efficiency with entropy [4], e.g. when a price series has high entropy, it has low redundancy and high information content, and hence there are no arbitrage opportunities. During periods of low entropy or inefficiency, bubbles can form. If we can measure the information entropy of a price series, we can create strategies that work better during periods of high or low entropy.

Before we can measure entropy, we have to encode our return series. We use return series instead of price series because of the need for stationarity — the author of AFML heavily emphasizes a fractionally differentiated price series to preserve the memory — something I will try soon.

After encoding the return series, a simple way to do it would be to count the number of unique bits needed to represent the price series, essentially creating a dictionary. Entropy is also a measure of compression, and if many bits are needed to explain a price series, the higher the entropy of a price series.

To actually encode and calculate entropy turned out to be quite hard without the flexibility of Python, so I implemented it on the platform indirectly using kurtosis.

Kurtosis is a measure of how ‘fat’ the tails of a distribution are. The higher the kurtosis, the more values are centred around the mean. If we were to encode the price series of a stock, one with higher kurtosis would likely have lower entropy.

Note that the kurtosis formula below measures excess kurtosis, so if you are calculating kurtosis from scratch, you will need to minus off 3, which is the kurtosis of a normal distribution.