Wow! Backtest RSI Crossover Strategy in Elasticsearch
By: Wai Tak Wong
Tags: Dataanalys, Python Programming, Data Science, Finance, Elasticsearch

Sometimes, you may have some crazy ideas and want to test your trading strategy. By using historical market data for backtesting, you can assess risk, profit, and measure performance to gain confidence to use the trading strategy in the future. Of course, you have no problem if you are a skilled programmer. However, you need to write a complex and heavy-loaded program to handle data analysis after the data is retrieved. Due to the powerful capabilities of Elasticsearch in search, data analysis, and machine learning, it is worth writing a series of articles about Elasticsearch backtesting different trading strategies for your reference.

In my previous article, RSI BB or RSI & BB? Easy, let’s test it with Elasticsearch, we have introduced how to use Elasticsearch to implement Relative Strength Index (RSI) indicator. This article will continue to make a simple tool to backtest the RSI trading Strategy in Elasticsearch. Since most of the work is done in the Elasticsearch cluster, the client will not have a heavy workload. Elasticsearch will provide buy or sell signals and then integrated them with a simple Python program to further analyze the signals and generate a report. Readers are advised to take a quick look at my previous article to understand RSI and the implementation details of using Elasticsearch.

RSI is a momentum indicator that provides information about price changes to support opportunities for buying and selling assets. The price changes are converted into two types of data, the recent total gains, and the recent total losses, usually with a period of 14. The RSI equation can be written as follows, where SMAgain,n,1 and SMAloss,n,1 are the total recent moving average of gains and the total recent moving average of losses. Corresponding to Elasticsearch SMA function with window n, it needs to shift 1 data to the right to include the current data.

The RSI crossover strategy defines the crossover of RSI at a specified value to indicate overbought (>= 70) and oversold (<= 30) signals. For other RSI values, be patient and wait for the buy or sell signal. It is much easier to use graphs to observe changes in values. In this article, we try to apply backtesting to commission-free exchange-traded funds (ETFs) and focus on Elasticsearch as an analysis tool. The following example randomly selects “Fidelity International Multifactor ETF”. Its ticker symbol is FDEV. 10 more ETFs randomly selected will be run, and the final results will be shown later. The data is selected from the time range between 2021–01–15 and 2021–05–31 provided by IEX, Investors Exchange. In the chart below, the RSI is plotted together with the daily closing price. In the daily price curve, the RSI values over 70 are marked in red, and the RSI values less than 30 are marked in blue.

Here, we present a simple RSI crossover strategy and use Elasticsearch to show the implementation details.

◆ Assuming that it is restricted to buy and hold 1 share at a time, no transaction will occur until the held share is sold.

◆ Buy 1 share when RSI value <= 30.

◆ Sell 1 share when RSI value >= 70.

◆ At the end of the testing period, a hold share is cashed with the current price.

There are 4 blue dots and 10 red dots, but only two buy and two sell transactions are allowed based on the strategy. Let’s describe the implementation using Elasticsearch. Suppose there is an Elasticsearch index populated with data, and its data mapping used is the same as described in the previous paper. The following steps demonstrate the code of the REST API request body.

Collect all relevant documents through the search operation

Use a “bool” query with a “must” clause to collect documents with the symbol FDEV and the date between 2021–01–15 and 2021–05–31. Due to the computation of 14-trading days moving average, additional data is adjusted for 1 month (from 2020–12–15 to 2021–01–14).

{

"query": {

"bool": {

"must": [

{"range": {"date": {"gte": "2020-12-15", "lte": "2021-05-31"}}},

{"term": {"symbol": "FDEV"}}

]

}

},

Extract the close value of the fund

Use a “date_histogram” aggregation, named Backtest_RSI, with the parameter “field” as “date” and the parameter “interval” as “1d” to extract the prices of the fund each day. Then followed by an “average” aggregation, named Daily, to retrieve the closing price since the subsequent pipeline aggregation cannot directly use the document fields.

"aggs": {

"Backtest_RSI": {

"date_histogram": {

"field": "date",

"interval": "1d",

"format": "yyyy-MM-dd"

},

"aggs": {

"Daily": {

"avg": {"field": "close"}

},

Extract the date of the bucket

Because of the additional data, subsequent operations need to filter out the out-of-range portion later. A “min” aggregation named “DateStr” is to get the date of the bucket. In the Elasticsearch server, the date field is stored in Epoch time. The time unit is milliseconds, and the time zone is UTC.

"DateStr": {

"min": {"field": "date"}

},

Select the buckets with more than 1 document

To filter out the empty buckets (non-trading days), a “bucket_selector” aggregation, named SDaily, is used to select buckets with its document count greater than 0.

"SDaily": {

"bucket_selector": {

"buckets_path": {"count":"_count"},

"script": "params.count > 0"

}

},

Calculate Daily difference

Use a “derivative” aggregation, named Price_Diff, and use the parameter “buckets_path” to specify the daily closing value to calculate the difference from the previous value.

"Price_Diff": {

"derivative": {

"buckets_path": "Daily"

}

},

Determine the daily value is a gain or a loss relative to the previous data

Use two “bucket_script” aggregations, named Gain and Loss, and use the parameter “buckets_path” to specify the result of Diff aggregation to determine value. Both values are positive.

"Gain": {

"bucket_script": {

"buckets_path": {"Price_Diff": "Price_Diff"},

"script": "(params.Price_Diff > 0) ? params.Price_Diff : 0"

}

},

"Loss": {

"bucket_script": {

"buckets_path": {"Price_Diff": "Price_Diff"},

"script": "(params.Price_Diff < 0) ? -params.Price_Diff : 0"

}

},

Calculate the daily simple moving average of the total Gain and the total Loss

Use two “moving_fn” aggregations, named GainSMA and LossSMA, with the parameter window as 14 and the parameter “buckets_path” as Gain and Loss respectively. The parameter “shift” is set to 1 to include data for the current day and the past 13 trading days. SMA is calculated by using the unweighted average function (MovingFunctions.unweightedAvg).

"GainSMA": {

"moving_fn": {

"script": "MovingFunctions.unweightedAvg(values)", "window": 14, "buckets_path": "Gain", "shift":1

}

},

"LossSMA": {

"moving_fn": {

"script": "MovingFunctions.unweightedAvg(values)", "window": 14, "buckets_path": "Loss", "shift":1

}

},

Calculate RSI

Use a “bucket_script” aggregation, named RSI, with the parameter “buckets_path” to specify the results from GainSMA and LossSMA. Then the RSI indicator is calculated according to the equation in the script.

"RSI": {

"bucket_script": {

"buckets_path": {"GainSMA": "GainSMA", "LossSMA": "LossSMA"},

"script": "100 - 100/(1+params.GainSMA/params.LossSMA)"

}

},

Identify the type of the RSI value

Use a “bucket_script” aggregation named, RSI_Type, with the parameter “buckets_path” to specify the RSI value to determine the type. Set the type to 2 if RSI value >= 70. Set the type to 1 if RSI value <= 30. Set the type to 0 for other RSI values.

"RSI_Type": {

"bucket_script": {

"buckets_path": {"RSI": "RSI"}, "script": "params.RSI >= 70 ? 2 : params.RSI <= 30 ? 1 : 0"

}

},

Filter out the additional documents for output

Use a “bucket_selector” aggregation, named Buy_Sell_Signal, with the parameter “buckets_path” to specify “DateStr” and “RSI_Type” to select the correct buckets. The selection criteria are those buckets having the date on or after 2021–01–15 (the epoch time 1612137600000 in milliseconds) and RSI_Type = 1 (RSI >= 70) or RSI_Type = 2 (RSI <= 30).

"Buy_Sell_Signal": {

"bucket_selector": {

"buckets_path": {"DateStr": "DateStr", "RSI_Type": "RSI_Type"},

"script": "params.DateStr >= 1610697600000L && (params.RSI_Type == 1 || params.RSI_Type == 2)"

}

}