Stock Trading Strategy Utilizing the Distribution of Predicted Returns
By: Haris, Chariton
Tags: Machine Learning, Stock Market, Distribution

A. Introduction

On this article we explain one of the three novel contributions behind the paper “High-performance stock index trading using neural networks and trees”, which achieves very strong results applied to major stock market indices such as SP500 and Dow Jones. The paper introduces a trading strategy that utilizes the distribution of predicted returns in order to make better and more informed trading decisions.

The purpose of this article is to describe the spirit of the trading strategy. For the actual implementation, various choices as well as the strong results we refer the curious reader to the paper. Moreover, given the financial implications linked with the paper code is unfortunately not available. However, the process is straightforward to replicate but what is even more important is to understand the idea behind it.

We begin by stating the motivation of the problem, next describe the distribution of predicted returns, a key principle for our trading strategy.

B. Problem Motivation

Let’s first consider the problem setup. Assume we have a prediction model f, that could be any model from linear regression to deep neural networks, which given input data X produces a prediction y_hat, i.e. f(X) = y_hat . Common practice and wisdom is to act solely on that prediction. For example, if we are trying to predict the closing price (either daily, monthly or any other time internal) of SP500 (assume current price is $1300) and use any finely tuned model to do so by getting a prediction of $1350 or 3.86%, a rational investor would buy the underlying instrument (for example an ETF). If the current price of SP500 is $1400, the investor would sell short (return = - 3.57%). Hence, the investor acts solely on the information of that prediction.

Imagine now two other scenarios: current price is $1300 and model predicts $1310 (0.769%) or $1400 (7.69%). By the same principle, the investor would buy probably without much consideration. In all scenarios the predicted return is indeed positive, implying an increase, however, how confident are we in each of our model’s prediction? i.e. how often in our training data, (assuming daily prediction) we had returns similar to 0.769% and how many to 7.69%, in order for our model to learn? Clearly for daily prediction smaller changes occur far more often.

The most common way for training and also judging how good a model is, relies on the use of some error function (such as mean absolute or squared error). This is definitely not the key here since, quite a lot of information is suppressed by averaging. By treating all three predictions the same we are unilaterally assuming that our model possesses the same “ability” in each scenario. This leads us to the question of how “successful” (this needs to be defined either using high returns, or low standard deviation or the ratio of two (loosely speaking, Sharpe Ratio)) is the model each time it predicts returns of 0.7%, 3.87% or 7%? In order to answer such question we resort to the historical distribution of predicted returns which contains significant information about our model’s ability. The objective is to find where our model does the best and where the worst and act accordingly.

C. Distribution of Predicted Returns

This is nothing more than the collection of all the available predicted returns for as many time steps as possible. The next step is to find ways to extract the required information. One way to do so is to “slice” the distribution into bins (e.g. deciles, quintiles etc..) and consider the “success” of each bin separately. For the moment, assume the following histogram (Fig 1) represents predicted returns on a daily frequency (randomly generated 1000 draws using standard normal distribution Z(0,2)). Let’s “slice” the distribution into 10 bins or deciles. This effectively bundles together returns within specific ranges.

Fig 1. Sample Distribution of predicted returns using 1000 samples from Z(0,2) along with its deciles.

For this example the ranges are as follows (Table 1)

Table 1. Bins of the predicted return distribution along with the count number.

For example, if predicted return is -1.57% then it belongs to bin 5 and if the predicted return is 3% then it belongs to bin 8.

OK, so what is the big deal?

Well, instead of computing statistics on the entire distribution and losing information by averaging, we now have 10 bins on which we can compute statistics and actually learn useful information.

To begin with, we can actually see that a high prediction as 7% occurred only twice (in our sample data) out of 1000 predictions. Hence how confident can we be when the model predicts a 7% increase that it’s actually going to be correct vs 0.79% which actually occur more often? This implies that our model has been exposed to far more observations in bin 6 than in bin 10. Or what if two bins are really “successful” (for example in terms of accuracy or profitability) and three are not? In that case you would like to avoid the bad bins and utilize to the maximum extent the good ones (for example whenever the model predicts a return that falls on the bad bins do nothing and when it falls on the good bins “go all in”). However, by treating all predictions the same, such useful information is lost and this is what will ultimately hurt overall performance.

The important takeaway is to note that we have effectively divided our prediction model into ten different models, one whose predictions falls in bin 1, a second one which falls in bin 2 and so on. Then by computing any metric for each bin we are harnessing more information versus treating all prediction the same.

D. How does it actually work?

The process is simple and as follows

Slice the distribution of predicted returns into bins

2. Calculate metric of “success” (e.g. profitability) of each bin

3. Locate which bin corresponds to the new prediction

4. If the bin is “successful” then act/trade, otherwise do nothing.

In order to apply the above procedure we require some initial “burn-in” data. When we start applying our methodology for our first prediction, we only have one value so not really a distribution. As we get more and more predictions, our distribution starts to take shape. How long we should we wait until we have a proper distribution depends on the prediction task i.e. daily vs monthly vs weekly, among other factors. Moreover, there is no assumption on having fixed cut-off points (i.e. using deciles) this was the most reasonable action to begin. Future research should look into variable points.

Example Walk Through

Let’s walk through a made-up example. Assume we are at time t and we want to compute our next day return prediction. Since we are going to use the same data as before, step (1) has already been done (Table 1). Once we have our distribution it is pretty straightforward to calculate any q-tile. For step (2) we define our metric of success to be the profitability as measured by cumulative returns.

How do we compute it for each bin? A new prediction comes up, we find which bin it belongs and gets a corresponding bin. We repeat for each of those 1000 predicted returns, as they become available, and assign them to one bin. An example is given at Table 2. Once we compute the predicted return 501, we use the previous 500 observations of predicted returns to compute the deciles and assign a bin to the new observation. Next day we record the realized return. Hopefully the realized return aligns with our predicted return, at least in direction!

Table 2. Each new predicted return get assigned to a bin, according to the deciles computed up to that point. Accordingly, the next day we record the realized return. By doing so, we can compute, historical performance of each bin. r_hat is the predicted return and r is the realized return.

So now we have for each bin a return associated with and simply compute the cumulative returns for each bin. This is equivalent to saying we are treating all bins as separate (ten) portfolios. For example, after observation 503, for bin = 5 we have two returns (and any returns associated with bin 5 in the first 500 observations). If bin 5 has 73 returns, then we compute the cumulative return of that bin based on those 73 returns. As we have already mentioned, any other measure of “success” could be used, i.e. median/mean of returns or standard deviation or Sharpe Ratio.

Why is this important? By measuring the success of each bin we evaluate how good our model is with respect to that bin. We decompose our overall ability and get a more fine-grained view by inspecting the distribution.

Back to our example, following the above process we calculate the profitability of each bin given at Table 3.

Table 3. Profitability for each bin

After steps (1) & (2) we have the profitability of each bin. For step (3) we need to simply locate the bin of our new prediction. Assume that our next day return prediction is 1.83%, which belongs to bin 7 and for step (4) we see how profitable that bin is, 5%.

To summarize: our predicted return is 1.83% which belongs to bin 7 (1.59% < 1.83% <= 2.94%) with the profitability of that bin being 5%, which triggers a buy signal. Had the bin been 6 we would not trade given the negative return.

Conclusion

I hope the above are sufficient to understand the basic principle of why someone should use the (historical) distribution of predicted returns. That being said, there are a lot of moving parts that required further research. For example, currently the bins are utilized the historical data it would be nice to have some predictive ability for each bin, other ways to slice the distribution (why prefix cut-off points?) or other measure success for each bin.