Repository: freqtrade/freqtrade
Pull request: Document "quick" hyperopt of roi/stoploss and trailing stoploss
## Summary
Document the "quick" hyperopt mode - without modifying the default hyperopt template.

Works only for ROI / stoploss / trailing stoploss spaces (not for buy / sell).

closes #2231

**Note:** I hope this renders correctly in the docs (codeblock within the "tip"-box), otherwise we'll need to move this as it's own header.

Problem statement:
Hyperopting strategies without creation of Hyperopt class
_"Show me the money"_ 😄 

The user is looking at the samples of the strategies, for example in the freqtrade-strategies, in the @berlinguyinca dir. She sees a strategy with a very attractive name ('Quickie'), the approach implemented in it also looks promising and interesting. She decides to backtest it:
```
2019-09-06 22:07:40,082 - freqtrade.optimize.backtesting - INFO - Backtesting with data from 2019-05-25T00:00:00+00:00 up to 2019-09-05T21:50:00+00:00 (103 days)..
2019-09-06 22:07:40,082 - freqtrade.optimize.backtesting - INFO - Running backtesting for Strategy Quickie
Result for strategy Quickie
========================================================= BACKTESTING REPORT ========================================================
| pair      |   buy count |   avg profit % |   cum profit % |   tot profit BTC |   tot profit % | avg duration     |   profit |   loss |
|:----------|------------:|---------------:|---------------:|-----------------:|---------------:|:-----------------|---------:|-------:|
| AST/BTC   |          28 |          -1.98 |         -55.32 |      -0.02768960 |         -18.44 | 1 day, 2:08:00   |       24 |      4 |
| ETC/BTC   |          11 |          -1.20 |         -13.16 |      -0.00658750 |          -4.39 | 1 day, 4:33:00   |       10 |      1 |
| ETH/BTC   |          12 |          -1.18 |         -14.16 |      -0.00708750 |          -4.72 | 2 days, 1:27:00  |       11 |      1 |
| EOS/BTC   |          16 |          -1.92 |         -30.74 |      -0.01538538 |         -10.25 | 2 days, 3:55:00  |       14 |      2 |
| IOTA/BTC  |          13 |          -2.72 |         -35.31 |      -0.01767500 |         -11.77 | 2 days, 23:10:00 |       11 |      2 |
| LTC/BTC   |          11 |          -1.09 |         -11.99 |      -0.00600186 |          -4.00 | 2 days, 22:38:00 |       10 |      1 |
| MTH/BTC   |          17 |           1.12 |          18.98 |       0.00950000 |           6.33 | 7:40:00          |       17 |      0 |
| NCASH/BTC |          33 |           1.98 |          65.23 |       0.03264526 |          21.74 | 9:07:00          |       31 |      2 |
| TNT/BTC   |          22 |           0.08 |           1.82 |       0.00091250 |           0.61 | 22:52:00         |       21 |      1 |
| XMR/BTC   |          17 |           1.00 |          16.98 |       0.00850000 |           5.66 | 14:02:00         |       17 |      0 |
| XLM/BTC   |          13 |          -5.35 |         -69.58 |      -0.03482415 |         -23.19 | 3 days, 12:01:00 |        8 |      5 |
| XRP/BTC   |           7 |          -2.74 |         -19.16 |      -0.00958750 |          -6.39 | 2 days, 15:35:00 |        6 |      1 |
| TOTAL     |         200 |          -0.73 |        -146.42 |      -0.07328073 |         -48.81 | 1 day, 10:25:00  |      180 |     20 |
========================================================= SELL REASON STATS =========================================================
| Sell Reason   |   Count |
|:--------------|--------:|
| roi           |     180 |
| stop_loss     |      14 |
| force_sell    |       3 |
| sell_signal   |       3 |
====================================================== LEFT OPEN TRADES REPORT ======================================================
| pair    |   buy count |   avg profit % |   cum profit % |   tot profit BTC |   tot profit % | avg duration      |   profit |   loss |
|:--------|------------:|---------------:|---------------:|-----------------:|---------------:|:------------------|---------:|-------:|
| AST/BTC |           1 |          -9.84 |          -9.84 |      -0.00492710 |          -3.28 | 3 days, 3:50:00   |        0 |      1 |
| EOS/BTC |           1 |         -19.58 |         -19.58 |      -0.00979788 |          -6.53 | 22 days, 9:05:00  |        0 |      1 |
| LTC/BTC |           1 |         -21.98 |         -21.98 |      -0.01100186 |          -7.33 | 22 days, 9:50:00  |        0 |      1 |
| TOTAL   |           3 |         -17.13 |         -51.40 |      -0.02572684 |         -17.13 | 15 days, 23:35:00 |        0 |      3 |
```
-- the strategy is 'bad', it gives ~ -49% loss...

However, she knows that results are highly dependent on the ROI tables and the stoploss value set. And she sees it in the backtesting results -- most reasons for the sells are due to roi or stoploss conditions. So she decides to hyperopt the strategy for roi and stoploss...

In the current freqtrade approach, she needs to create a separate hyperopt file with hyperopt class, fill it with the methods. So she cannot simply iterate through a directory with strategies and adjust quickly roi/stoploss to her exchange/pairs/timeframe/timerange...

Assume, we could load a strategy as hyperopt 'as is', with small modifications (that would not prevent the user from using a strategy for backtesting and trading) and perform hyperoptimization without annoying actions needed for creation of the hyperopt file and class...

So she could simply write in that Quickie strategy (or her own custom strategy):

```
...
from freqtrade.strategy.interface import IStrategy
from freqtrade.optimize.hyperopt_interface import IHyperOpt
...

class Quickie(IStrategy, IHyperOpt):
...
// The strategy as it is
```

Then, she could run hyperoptimization:
```
$ freqtrade -c config_binance.json.example hyperopt --hyperopt-path user_data/strategies/berlinguyinca/ --customhyperopt Quickie --spaces roi stoploss -e 500 --print-all --hyperopt-loss OnlyProfitHyperOptLoss -i 5m
...
2019-09-06 21:13:34,619 - freqtrade.optimize.hyperopt - INFO - Hyperopting with data from 2019-05-25T00:00:00+00:00 up to 2019-09-05T21:50:00+00:00 (103 days)..
2019-09-06 21:13:34,932 - freqtrade.optimize.hyperopt - INFO - Found 4 CPU cores. Let's make them scream!
...
Best result:

   175/500:    703 trades. Avg profit  0.14%. Total profit  0.04970709 BTC (  99.31Σ%). Avg duration 539.2 mins. Objective: 0.66896

ROI table:
{0: 0.06384, 15: 0.04856, 59: 0.01609, 174: 0}
Stoploss: -0.09692
```
-- Voila, the strategy is not that bad...

Hints from issue discussion:
looks beautiful!

On Fri, Sep 6, 2019 at 7:23 PM hroff-1902 <notifications@github.com> wrote:

> *"Show me the money"* 😄
>
> The user is looking at the samples of the strategies, for example in the
> freqtrade-strategies, in the @berlinguyinca
> <https://github.com/berlinguyinca> dir. She sees a strategy with very
> attractive name ('Quickie'), the approach implemented in it also looks
> promising and interesting. She decides to backtest it:
>
> 2019-09-06 22:07:40,082 - freqtrade.optimize.backtesting - INFO - Backtesting with data from 2019-05-25T00:00:00+00:00 up to 2019-09-05T21:50:00+00:00 (103 days)..
>
> 2019-09-06 22:07:40,082 - freqtrade.optimize.backtesting - INFO - Running backtesting for Strategy Quickie
>
> Result for strategy Quickie
>
> ========================================================= BACKTESTING REPORT ========================================================
>
> | pair      |   buy count |   avg profit % |   cum profit % |   tot profit BTC |   tot profit % | avg duration     |   profit |   loss |
>
> |:----------|------------:|---------------:|---------------:|-----------------:|---------------:|:-----------------|---------:|-------:|
>
> | AST/BTC   |          28 |          -1.98 |         -55.32 |      -0.02768960 |         -18.44 | 1 day, 2:08:00   |       24 |      4 |
>
> | ETC/BTC   |          11 |          -1.20 |         -13.16 |      -0.00658750 |          -4.39 | 1 day, 4:33:00   |       10 |      1 |
>
> | ETH/BTC   |          12 |          -1.18 |         -14.16 |      -0.00708750 |          -4.72 | 2 days, 1:27:00  |       11 |      1 |
>
> | EOS/BTC   |          16 |          -1.92 |         -30.74 |      -0.01538538 |         -10.25 | 2 days, 3:55:00  |       14 |      2 |
>
> | IOTA/BTC  |          13 |          -2.72 |         -35.31 |      -0.01767500 |         -11.77 | 2 days, 23:10:00 |       11 |      2 |
>
> | LTC/BTC   |          11 |          -1.09 |         -11.99 |      -0.00600186 |          -4.00 | 2 days, 22:38:00 |       10 |      1 |
>
> | MTH/BTC   |          17 |           1.12 |          18.98 |       0.00950000 |           6.33 | 7:40:00          |       17 |      0 |
>
> | NCASH/BTC |          33 |           1.98 |          65.23 |       0.03264526 |          21.74 | 9:07:00          |       31 |      2 |
>
> | TNT/BTC   |          22 |           0.08 |           1.82 |       0.00091250 |           0.61 | 22:52:00         |       21 |      1 |
>
> | XMR/BTC   |          17 |           1.00 |          16.98 |       0.00850000 |           5.66 | 14:02:00         |       17 |      0 |
>
> | XLM/BTC   |          13 |          -5.35 |         -69.58 |      -0.03482415 |         -23.19 | 3 days, 12:01:00 |        8 |      5 |
>
> | XRP/BTC   |           7 |          -2.74 |         -19.16 |      -0.00958750 |          -6.39 | 2 days, 15:35:00 |        6 |      1 |
>
> | TOTAL     |         200 |          -0.73 |        -146.42 |      -0.07328073 |         -48.81 | 1 day, 10:25:00  |      180 |     20 |
>
> ========================================================= SELL REASON STATS =========================================================
>
> | Sell Reason   |   Count |
>
> |:--------------|--------:|
>
> | roi           |     180 |
>
> | stop_loss     |      14 |
>
> | force_sell    |       3 |
>
> | sell_signal   |       3 |
>
> ====================================================== LEFT OPEN TRADES REPORT ======================================================
>
> | pair    |   buy count |   avg profit % |   cum profit % |   tot profit BTC |   tot profit % | avg duration      |   profit |   loss |
>
> |:--------|------------:|---------------:|---------------:|-----------------:|---------------:|:------------------|---------:|-------:|
>
> | AST/BTC |           1 |          -9.84 |          -9.84 |      -0.00492710 |          -3.28 | 3 days, 3:50:00   |        0 |      1 |
>
> | EOS/BTC |           1 |         -19.58 |         -19.58 |      -0.00979788 |          -6.53 | 22 days, 9:05:00  |        0 |      1 |
>
> | LTC/BTC |           1 |         -21.98 |         -21.98 |      -0.01100186 |          -7.33 | 22 days, 9:50:00  |        0 |      1 |
>
> | TOTAL   |           3 |         -17.13 |         -51.40 |      -0.02572684 |         -17.13 | 15 days, 23:35:00 |        0 |      3 |
>
>
> -- the strategy is 'bad', it gives ~ -49% loss...
>
> However, she knows that results are highly dependent on the ROI tables and
> the stoploss value set. And she sees it in the backtesting results -- most
> results for the sells are due to roi or stoploss conditions. So she decides
> to hyperopt the strategy for roi and stoploss...
>
> In the current freqtrade approach, she needs to create a separate hyperopt
> file with hyperopt class, fill it with the methods. So she cannot simply
> iterate through a directory with strategies and adjust quickly roi/stoploss
> to her exchange/pairs/timeframe/timerange...
>
> Assume, we could load a strategy as hyperopt 'as is', with small
> modifications (that would not prevent the user from using a strategy for
> backtesting and trading) and perform hyperoptimization without annoying
> actions needed for creation of the hyperopt file and class...
>
> So she could simply write in that Quickie strategy (or her own custom
> strategy):
>
> ...
>
> from freqtrade.strategy.interface import IStrategy
>
> from freqtrade.optimize.hyperopt_interface import IHyperOpt
>
> ...
>
>
>
> class Quickie(IStrategy, IHyperOpt):
>
> ...
>
> // The strategy as it is
>
>
> Then, she could run hyperoptimization:
>
> $ freqtrade -c config_binance.json.example hyperopt --hyperopt-path user_data/strategies/berlinguyinca/ --customhyperopt Quickie --spaces roi stoploss -e 500 --print-all --hyperopt-loss OnlyProfitHyperOptLoss -i 5m
>
> ...
>
> 2019-09-06 21:13:34,619 - freqtrade.optimize.hyperopt - INFO - Hyperopting with data from 2019-05-25T00:00:00+00:00 up to 2019-09-05T21:50:00+00:00 (103 days)..
>
> 2019-09-06 21:13:34,932 - freqtrade.optimize.hyperopt - INFO - Found 4 CPU cores. Let's make them scream!
>
> ...
>
> Best result:
>
>
>
>    175/500:    703 trades. Avg profit  0.14%. Total profit  0.04970709 BTC (  99.31Σ%). Avg duration 539.2 mins. Objective: 0.66896
>
>
>
> ROI table:
>
> {0: 0.06384, 15: 0.04856, 59: 0.01609, 174: 0}
>
> Stoploss: -0.09692
>
>
> -- Voila, the strategy is not that bad...
>
> —
> You are receiving this because you were mentioned.
> Reply to this email directly, view it on GitHub
> <https://github.com/freqtrade/freqtrade/issues/2231?email_source=notifications&email_token=AAAD73FZIA2VW2OLRXNMARTQIKU4VA5CNFSM4IUMJGJ2YY3PNVWWK3TUL52HS4DFUVEXG43VMWVGG33NNVSW45C7NFSM4HJ4G7EA>,
> or mute the thread
> <https://github.com/notifications/unsubscribe-auth/AAAD73G53YKD24IB3ER6FUDQIKU4VANCNFSM4IUMJGJQ>
> .
>


-- 
------------------------------------------------------------
Lead Developer - Fiehnlab, UC Davis

gert wohlgemuth

work:
http://fiehnlab.ucdavis.edu/staff/wohlgemuth

phone:
530 665 9477

coding blog:
http://codingandmore.blogspot.com

linkedin:
http://www.linkedin.com/profile/view?id=28611299&trk=tab_pro

See https://github.com/freqtrade/freqtrade/pull/2232#issuecomment-529083628 for my 2 cents on the above approach (i like the idea in general though)


## An alternative idea to facilitate this:
To facilitate simple Hyperopting, you should instead try to use `StrategyResolver` to import a custom strategy and run / use the strategies `populate_*` methods.
This will obviously require to remove the constraint to only allow DefaultStrategy - but IF you can get that to work that would be best

sample call:
`freqtrade --strategy MyCoolStrategy hyperopt -e 100 --space ROI,stoploss`

This approach would not require **any** change to either strategy interface, nor to the strategy itself - however there is one caveat i ran into previously (which is causing the contraint to `DefaultStrategy`:

when running with anything other than `--j 1` - You get unpickle errors.
I think by manipulating the functions `__dict__` (basically faking a valid import location) this could be fixed - but i gave up on this idea since it cost a ton of time without much benefit - since you needed to write a custom hyperopt file anyway, and copy/pasting `populate_indicators()` was a small hurdle. 

### possible alternative for "quick hyperopt":

Maybe, for the above "special" mode - you could fix this by running `populate_*` methods before invoking parallelism - since you won't change buy/sell signals anyway (only ROI and stoploss change).

This would lead to the following approach:
* quick hyperopt - loads custom strategy and works only for ROI and Stoploss
* real hyperopt - requires custom hyperopt file (will require this anyway - since you cannot know buy/sell space that fits the strategy's indicators) - and is constraint to DefaultStrategy (to avoid the above problem).

I'm currently not clear (not that deep into hyperopt at the moment) if that will require a seperate HyperoptClass, or if it would work as is - however i think this should be feasible.


Sorry this became long ... just my 2 cents ;) 
Let's first agree on functionality similar to one described above -- is it useful, should we think of it as of a target or just through it away as useless. 

The implementations may differ and I proposed the quick and dirty approach which is achievable with the current freqtrade codebase with minimal changes to the current freqtrade codebase (the PR is very small). It may have (and has, as you highlighted) some caveats that are to be mitigated in the discussions and design/implementation changes.

With this approach as a whole, we could automatically iterate (maybe with github repetitive tasks scheduled) through all the strategies every month, fetch stoploss and roi tables that work with default config (exchange, pair, timeframe, etc) best in the current market conditions for last, say, 3 months), and update strategies. The users (especially novices that just installed the bot and look into the sample strategies) will obtain a set of better working strategies. This can be done every 1st day of each month or before each (or some) master release. Better user experience, less questions 'why all strategies are so bad'...

your idea (using --strategy for hyperopting) is also fine, I will think about all this

I'm not sure about usefulness - it would assume that buy/sell signals are great as they are - which may (or may not) be true.


I'm not against, this, however I would not go for the quick and dirty approach - it would mean to add iHyperopt to all strategies (see PR for reasons why this is bad), and that's a design decision i would not add lightly (since it's very difficult to revert in the future, once people have used the new spec as base for their files/strategies).

Also i think the approach to use hyperopt for the validation of strategies is wrong.

If you do some automation like that, then i would use backtesting, not hyperopt, and simply iterate over all strategies - this will determine the best strategy for current market conditions.

If you use hyperopt and change ROI/stoploss, it would result in information difficult to understand for new users since the strategy may change every month.

And worse, assuming someone uses strategies from the repo "as they are" since they work for a specific market, the changes done by hyperopt may break this niche strategy, so from that regard, having "consistency" (non-changing) in the strategy is good.


### Something slightly different 
(but relevant if you consider adding IHyperopt to strategies)

We should splitting hyperopt dependencies in setup.py (so you install them with `pip install -e freqtrade[hyperopt]`.
"regular" trading operations don't need to install skopt - and i assume "trading" to be the main purpose (you develop a strategy once, probably on your pc, maybe tune it a bit, but then start the bot and leave it running for a longer time on a server).

Removing this would simplify deployment to low-powered machines (RPI) - since skopt (and scipy) are a pain to install there. This may allow us to lower min requirements ... (which however will still depend strongly on users strategies and indicators used).


> ### Something slightly different
> (but relevant if you consider adding IHyperopt to strategies)
> 
> We should splitting hyperopt dependencies in setup.py (so you install them with `pip install -e freqtrade[hyperopt]`.
> "regular" trading operations don't need to install skopt - and i assume "trading" to be the main purpose (you develop a strategy once, probably on your pc, maybe tune it a bit, but then start the bot and leave it running for a longer time on a server).
> 
> Removing this would simplify deployment to low-powered machines (RPI) - since skopt (and scipy) are a pain to install there. This may allow us to lower min requirements ... (which however will still depend strongly on users strategies and indicators used).

Excellent! 👍 💯 

will add this as step to the "road to pip" issue - since it's not really relevant for this issue.

> This will obviously require to remove the constraint to only allow DefaultStrategy - but IF you can get that to work that would be best

Done, Sir! (see #2235)

I think we can close this issue (or at least relabel it to "documentation").

this is now possible with the latest develop (just tested):

```
# Have a working strategy at hand.
freqtrade new-hyperopt --hyperopt EmptyHyperopt

freqtrade hyperopt --hyperopt EmptyHyperopt --spaces roi stoploss trailing --strategy MyWorkingStrategy --config whateverconfig.json -e 100
# Sit back and hope your pc does not catch fire
```

Since the created hyperopt does not include `populate_buy()` nor `populate_sell()` methods (nor `populate_indicators()`) - i get
```
2019-12-05 20:06:05,704 - freqtrade.resolvers.hyperopt_resolver - WARNING - Hyperopt class does not provide populate_indicators() method. Using populate_indicators from the strategy.
2019-12-05 20:06:05,704 - freqtrade.resolvers.hyperopt_resolver - WARNING - Hyperopt class does not provide populate_buy_trend() method. Using populate_buy_trend from the strategy.
2019-12-05 20:06:05,704 - freqtrade.resolvers.hyperopt_resolver - WARNING - Hyperopt class does not provide populate_sell_trend() method. Using populate_sell_trend from the strategy.
```

We maybe should consider to change the log-level of this message from Warning to Info  but technically this is working perfectly fine.
Ah yeah, right, closing it.
👍 

The docs should probably somehow mention this hidden knowledge...

I also have some ideas for "quick hyperopt" you mentioned above, but in some different form...

Commit message: Document "quick" hyperopt of roi/stoploss and trailing stoploss

Patch to review:
diff --git a/docs/hyperopt.md b/docs/hyperopt.md
index 0661896a0e0..befd448bb1b 100644
--- a/docs/hyperopt.md
+++ b/docs/hyperopt.md
@@ -50,6 +50,16 @@ Rarely you may also need to override:
 * `stoploss_space` - for custom stoploss optimization (if you need the range for the stoploss parameter in the optimization hyperspace that differs from default)
 * `trailing_space` - for custom trailing stop optimization (if you need the ranges for the trailing stop parameters in the optimization hyperspace that differ from default)
 
+!!! Tip "Quickly optimize ROI, stoploss and trailing stoploss"
+    You can quickly optimize the spaces `roi`, `stoploss` and `trailing_stoploss` without changing anything from the default hyperopt template by relying on your strategy to do most of the calculations.
+
+    ``` python
+    # Have a working strategy at hand.
+    freqtrade new-hyperopt --hyperopt EmptyHyperopt
+
+    freqtrade hyperopt --hyperopt EmptyHyperopt --spaces roi stoploss trailing --strategy MyWorkingStrategy --config config.json -e 100
+    ``` 
+
 ### 1. Install a Custom Hyperopt File
 
 Put your hyperopt file into the directory `user_data/hyperopts`.


Code review comments:
diff_hunk: @@ -50,6 +50,16 @@ Rarely you may also need to override:
 * `stoploss_space` - for custom stoploss optimization (if you need the range for the stoploss parameter in the optimization hyperspace that differs from default)
 * `trailing_space` - for custom trailing stop optimization (if you need the ranges for the trailing stop parameters in the optimization hyperspace that differ from default)
 
+!!! Tip "Quickly optimize ROI, stoploss and trailing stoploss"
+    You can quickly optimize the spaces `roi`, `stoploss` and `trailing_stoploss` without changing anything from the default hyperopt template by relying on your strategy to do most of the calculations.
path: docs/hyperopt.md
text: @user1:
* the space is named 'trailing'
* maybe: "without changing anything (i.e. without creation of a "complete" Hyperopt class with dimensions, parameters, triggers and guards, as described in this document) from the default hyperopt template by relying..."