Stop Loss, Trailing Stop, or Take Profit? 2 Million Backtests Shed Light
By: Oleg Polakow
Tags: Trading, Cryptocurrency, Stop Loss, Backtesting, Optimization

LARGE-SCALE BACKTESTING IN 5 MINUTES

In this article, we will utilize large-scale backtesting with vectorbt to explore the performance of the most common stop signals for different cryptocurrencies, time periods, and stop values.

Photo by FLY:D on Unsplash

A trading strategy is just a grain of sand when compared to the whole universe of possible strategies; only the big picture can reveal its quality.

Methodology

Our goal is to utilize large-scale backtesting to compare the performance of trading with and without stop loss (SL), trailing stop (TS), and take profit (TP) signals. To make this attempt representative, we will run a huge number of experiments across three different dimensions: instruments, time, and hyperparameters:

First, we will pick 10 cryptocurrencies by market capitalization (except stablecoins such as USDT) and fetch 3 years of their daily pricing data. In particular, we aim at backtesting the time period from 2018 to 2021 as it contains periods of sharp price drops (e.g., corrections due to ATH in December 2017 and coronavirus in March 2020) as well as surges (ATH in December 2020) — this keeps things balanced.

For each instrument, we will split this time period into 400 smaller (and overlapping) time windows, each 6 months long. We will run our tests on each of these windows to account for different market regimes.

For each instrument and time window, we will then generate an entry signal at the very first bar and find an exit signal according to the stop configuration. We will test 100 stop values with a 1% increment and compare the performance of each one to that of trading randomly and holding within this particular time window.

In total, we will conduct 2,000,000 backtests.

Setup

All we need is Jupyter Notebook/Lab with Python ≥ 3.6, yfinance, vectorbt, and packages required by them. We will use yfinance to download pricing data, and vectorbt to both run 2 million backtests in under 5 minutes and analyze the results visually.

vectorbt is a next-generation backtesting library for Python that applies various backtesting and data science techniques to technical analysis. The way it works is by representing trading data — from time series to order records — as nd-arrays, and processing them using NumPy and Numba. This in turn enables use cases such as blazingly fast hyperparameter optimization, which is otherwise mainly done using distributed and cloud computing. Another advantage is integration of Plotly and ipywidgets to display interactive charts and dashboards right in the Jupyter notebook.

Define parameters

The first step is to define the parameters of the analysis pipeline. As discussed above, we will backtest 3 years of pricing data, 400 time windows, 10 cryptocurrencies, and 100 stop values. We will also set fees and slippage both to 0.25% and initial capital to $100 (the amount per se doesn’t matter, but it must be the same for all assets to be comparable). Feel free to change any parameter of interest.

Start date 2018-01-01 00:00:00

End date 2021-01-01 00:00:00

Time period (days) 1096

Assets 10

Window length 180 days, 0:00:00

Windows 400

Exit types 5

Stop values 100

Tests per asset 200000

Tests per window 5000

Tests per exit type 400000

Tests per stop type and value 4000

Tests total 2000000

dtype: object

Our configuration yields sample sizes with enough statistical power to analyze four variables: assets (200k tests per asset), time (5k tests per time window), exit types (400k tests per exit type), and stop values (4k tests per stop type and value). Similar to how Tableau handles dimensions and measures, we will be able to group our performance by each of these variables, but we will mainly focus on 5 exit types: SL exits, TS exits, TP exits, random exits, and holding exits (placed at the last bar).

Download data

Getting daily pricing data of each cryptocurrency is straightforward using yfinance:

dict_keys(['BTC-USD', 'ETH-USD', 'XRP-USD', 'BCH-USD', 'LTC-USD', 'BNB-USD', 'EOS-USD', 'XLM-USD', 'XMR-USD', 'ADA-USD'])

(1083, 5)

The dictionary ohlcv_by_symbol now contains OHLCV data by cryptocurrency name. Each DataFrame has 1083 rows (days) and 5 columns (O, H, L, C, and V). You can plot a DataFrame as follows:

OHLCV of BTC-USD pair

Since assets are one of the dimensions we want to analyze, vectorbt expects us to pack them as columns into a single DataFrame and label them accordingly. To do so, we simply swap assets and features to get a dictionary of DataFrames (with assets now as columns) keyed by feature name, such as “Open”.

dict_keys(['Open', 'Low', 'High', 'Close', 'Volume'])

(1083, 10)

Generate time windows

Next, we will move a 6-month sliding window over the whole time period and take 400 “snapshots” of each price DataFrame within this window. Each snapshot will correspond to a subset of data that should be independently backtested. As with assets and other variables, snapshots also need to be stacked horizontally as columns. As a result, we will get 180 rows (window length in days) and 4000 columns (10 assets x 400 windows); that is, one column will correspond to the price of one asset within one particular time window.

(180, 4000)

A nice feature of vectorbt is that it makes use of hierarchical indexing to store valuable information on each backtest. It also ensures that this column hierarchy is preserved across the whole backtesting pipeline — from signal generation to performance modeling — and can be extended easily. Currently, our columns have the following hierarchy:

MultiIndex([

('BTC-USD', '2017-12-31', '2018-06-28'),

('BTC-USD', '2018-01-02', '2018-06-30'),

('BTC-USD', '2018-01-05', '2018-07-03'),

...

('ADA-USD', '2020-06-16', '2020-12-12'),

('ADA-USD', '2020-06-19', '2020-12-15'),

('ADA-USD', '2020-06-21', '2020-12-17')

], names=[

'symbol',

'range_start',

'range_end'

], length=4000)

This multi-index captures three parameters: the symbol, the start date of the time window, and its end date. Later, we will extend this multi-index with exit types and stop values such that each of the 2 million backtests has its own price series.

Generate entry signals

In contrast to most other backtesting libraries, signals are not stored as a signed integer array, but they are split into two boolean arrays: entries and exits, which makes manipulation a lot easier.

At the beginning of each time window, let’s generate an entry signal indicating a buy order. The data frame will have the same shape, index, and columns as that of price so that vectorbt can link their elements together.

(180, 4000)

Generate exit signals

For each of the entry signals we generated, we will find an exit signal according to our 5 exit types: SL, TS, TP, random, and holding. We will also concatenate their DataFrames into a single (huge) DataFrame with 180 rows and 2,000,000 columns, each representing a separate backtest. Since exit signals are boolean, their memory footprint is tolerable.

Let’s generate exit signals according to stop conditions first. We want to test 100 different stop values with a 1% increment, starting from 1% and ending with 100% (i.e., find a timestamp where the price exceeds the entry price by 100%). Usually, when OHLC data is checked against such conditions, the position is closed at (or shortly after) the time of hitting the particular stop, but we will simplify things and use the “Close” price to exit any position.

(180, 400000) (180, 400000) (180, 400000)

This also extended our column hierarchy with a new column level indicating the stop value, we only have to make it consistent across all DataFrames:

MultiIndex([

(0.01, 'BTC-USD', '2017-12-31', '2018-06-28'),

(0.01, 'BTC-USD', '2018-01-02', '2018-06-30'),

(0.01, 'BTC-USD', '2018-01-05', '2018-07-03'),

...

( 1.0, 'ADA-USD', '2020-06-16', '2020-12-12'),

( 1.0, 'ADA-USD', '2020-06-19', '2020-12-15'),

( 1.0, 'ADA-USD', '2020-06-21', '2020-12-17')

], names=[

'stop_value',

'symbol',

'range_start',

'range_end'

], length=400000)

One major feature of vectorbt is that it places a strong focus on data science, and so it allows us to apply popular analysis tools to almost any part of the backtesting pipeline. For example, let’s explore how the number of exit signals depends upon the stop type and value:

SL 0.434195

TS 0.590803

TP 0.514545

Name: avg_num_signals, dtype: float64

Average number of signals by exit type and stop value

We see that TS is by far the most occurring exit signal. The SL and TP curves come hand in hand up to the stop value of 50% and then diverge in favor of TP. While it might seem that bulls are mostly in charge, especially for bigger price movements, remember that it is much easier to post a 50% profit than a 50% loss because the latter requires a 100% profit to recover; thus, negative downward spikes seem to dominate small to medium price movements (and shake out weak hands potentially). These are well-known cryptocurrency dynamics.

To simplify the analysis that follows, we should ensure that each column has at least one exit signal to close the position, which means that if a column has no exit signal now, it should get one at the last timestamp. This is done by combining the stop exits with the last-bar exit using the OR rule and selecting the one that comes first: