Repository: freqtrade/freqtrade

Issue:
ValueError: 'pair' is both an index level and a column label, which is ambiguous
<!-- 
Have you searched for similar issues before posting it?

If you have discovered a bug in the bot, please [search our issue tracker](https://github.com/freqtrade/freqtrade/issues?q=is%3Aissue). 
If it hasn't been reported, please create a new issue.

Please do not use bug reports to request new features.
-->

## Describe your environment

  * Operating system: Ubuntu 20.04
  * Python Version: 3.7.4 (`python -V`)
  * CCXT version: ccxt==1.32.7 (`pip freeze | grep ccxt`)
  * Freqtrade Version: develop-6ca057e2 (`freqtrade -V` or `docker-compose run --rm freqtrade -V` for Freqtrade running in docker)
  
Note: All issues other than enhancement requests will be closed without further comment if the above template is deleted or not filled out.

## Describe the problem:

*Explain the problem you have encountered*
Things just blow up when I enable edge.

### Steps to reproduce:

  1. _____
  2. _____
  3. _____
  
### Observed Results:

  * What happened?
  * What did you expect to happen?

### Relevant code exceptions or logs

Note: Please copy/paste text of the messages, no screenshots of logs please.

  ```
Jul 27 19:27:35 knutas-backend freqtrade[1783272]: 2020-07-27 19:27:35,606 - freqtrade.commands.trade_commands - ERROR - 'pair' is both an index level and a column label, which is ambiguous.
Jul 27 19:27:35 knutas-backend freqtrade[1783272]: 2020-07-27 19:27:35,606 - freqtrade.commands.trade_commands - ERROR - Fatal exception!
Jul 27 19:27:35 knutas-backend freqtrade[1783272]: Traceback (most recent call last):
Jul 27 19:27:35 knutas-backend freqtrade[1783272]:   File "/home/knutas/anaconda3/envs/freqtrade/lib/python3.7/site-packages/freqtrade-develop_6ca057e2-py3.7.egg/freqtrade/commands/trade_commands.py", line 19, in start_trading
Jul 27 19:27:35 knutas-backend freqtrade[1783272]:     worker = Worker(args)
Jul 27 19:27:35 knutas-backend freqtrade[1783272]:   File "/home/knutas/anaconda3/envs/freqtrade/lib/python3.7/site-packages/freqtrade-develop_6ca057e2-py3.7.egg/freqtrade/worker.py", line 34, in __init__
Jul 27 19:27:35 knutas-backend freqtrade[1783272]:     self._init(False)
Jul 27 19:27:35 knutas-backend freqtrade[1783272]:   File "/home/knutas/anaconda3/envs/freqtrade/lib/python3.7/site-packages/freqtrade-develop_6ca057e2-py3.7.egg/freqtrade/worker.py", line 51, in _init
Jul 27 19:27:35 knutas-backend freqtrade[1783272]:     self.freqtrade = FreqtradeBot(self._config)
Jul 27 19:27:35 knutas-backend freqtrade[1783272]:   File "/home/knutas/anaconda3/envs/freqtrade/lib/python3.7/site-packages/freqtrade-develop_6ca057e2-py3.7.egg/freqtrade/freqtradebot.py", line 87, in __init__
Jul 27 19:27:35 knutas-backend freqtrade[1783272]:     self.active_pair_whitelist = self._refresh_active_whitelist()
Jul 27 19:27:35 knutas-backend freqtrade[1783272]:   File "/home/knutas/anaconda3/envs/freqtrade/lib/python3.7/site-packages/freqtrade-develop_6ca057e2-py3.7.egg/freqtrade/freqtradebot.py", line 213, in _refresh_active_whitelist
Jul 27 19:27:35 knutas-backend freqtrade[1783272]:     self.edge.calculate()
Jul 27 19:27:35 knutas-backend freqtrade[1783272]:   File "/home/knutas/anaconda3/envs/freqtrade/lib/python3.7/site-packages/freqtrade-develop_6ca057e2-py3.7.egg/freqtrade/edge/edge_positioning.py", line 150, in calculate
Jul 27 19:27:35 knutas-backend freqtrade[1783272]:     self._cached_pairs = self._process_expectancy(trades_df)
Jul 27 19:27:35 knutas-backend freqtrade[1783272]:   File "/home/knutas/anaconda3/envs/freqtrade/lib/python3.7/site-packages/freqtrade-develop_6ca057e2-py3.7.egg/freqtrade/edge/edge_positioning.py", line 307, in _process_expectancy
Jul 27 19:27:35 knutas-backend freqtrade[1783272]:     df = results.groupby(['pair', 'stoploss'])[['profit_abs', 'trade_duration']].agg(
Jul 27 19:27:35 knutas-backend freqtrade[1783272]:   File "/home/knutas/anaconda3/envs/freqtrade/lib/python3.7/site-packages/pandas/core/frame.py", line 5810, in groupby
Jul 27 19:27:35 knutas-backend freqtrade[1783272]:     observed=observed,
Jul 27 19:27:35 knutas-backend freqtrade[1783272]:   File "/home/knutas/anaconda3/envs/freqtrade/lib/python3.7/site-packages/pandas/core/groupby/groupby.py", line 410, in __init__
Jul 27 19:27:35 knutas-backend freqtrade[1783272]:     mutated=self.mutated,
Jul 27 19:27:35 knutas-backend freqtrade[1783272]:   File "/home/knutas/anaconda3/envs/freqtrade/lib/python3.7/site-packages/pandas/core/groupby/grouper.py", line 594, in get_grouper
Jul 27 19:27:35 knutas-backend freqtrade[1783272]:     obj._check_label_or_level_ambiguity(gpr, axis=axis)
Jul 27 19:27:35 knutas-backend freqtrade[1783272]:   File "/home/knutas/anaconda3/envs/freqtrade/lib/python3.7/site-packages/pandas/core/generic.py", line 1647, in _check_label_or_level_ambiguity
Jul 27 19:27:35 knutas-backend freqtrade[1783272]:     raise ValueError(msg)
Jul 27 19:27:35 knutas-backend freqtrade[1783272]: ValueError: 'pair' is both an index level and a column label, which is ambiguous.
  ```



Discussion:
how does the edge section of your configuration look like?

using the default - i'm unable to recreate this (and honestly, i don't plan on trying every posible constellation if you got a "breaking" configuration right at hand :laughing: ).

Resolving patch:
diff --git a/freqtrade/edge/edge_positioning.py b/freqtrade/edge/edge_positioning.py
--- a/freqtrade/edge/edge_positioning.py
+++ b/freqtrade/edge/edge_positioning.py
@@ -281,8 +281,8 @@ def _process_expectancy(self, results: DataFrame) -> Dict[str, Any]:
         #
         # Removing Pumps
         if self.edge_config.get('remove_pumps', False):
-            results = results.groupby(['pair', 'stoploss']).apply(
-                lambda x: x[x['profit_abs'] < 2 * x['profit_abs'].std() + x['profit_abs'].mean()])
+            results = results[results['profit_abs'] < 2 * results['profit_abs'].std()
+                              + results['profit_abs'].mean()]
         ##########################################################################
 
         # Removing trades having a duration more than X minutes (set in config)


Test patch:
diff --git a/tests/edge/test_edge.py b/tests/edge/test_edge.py
--- a/tests/edge/test_edge.py
+++ b/tests/edge/test_edge.py
@@ -409,3 +409,98 @@ def get_fee(*args, **kwargs):
     final = edge._process_expectancy(trades_df)
     assert len(final) == 0
     assert isinstance(final, dict)
+
+
+def test_process_expectancy_remove_pumps(mocker, edge_conf, fee,):
+    edge_conf['edge']['min_trade_number'] = 2
+    edge_conf['edge']['remove_pumps'] = True
+    freqtrade = get_patched_freqtradebot(mocker, edge_conf)
+
+    freqtrade.exchange.get_fee = fee
+    edge = Edge(edge_conf, freqtrade.exchange, freqtrade.strategy)
+
+    trades = [
+        {'pair': 'TEST/BTC',
+         'stoploss': -0.9,
+         'profit_percent': '',
+         'profit_abs': '',
+         'open_time': np.datetime64('2018-10-03T00:05:00.000000000'),
+         'close_time': np.datetime64('2018-10-03T00:10:00.000000000'),
+         'open_index': 1,
+         'close_index': 1,
+         'trade_duration': '',
+         'open_rate': 17,
+         'close_rate': 15,
+         'exit_type': 'sell_signal'},
+
+        {'pair': 'TEST/BTC',
+         'stoploss': -0.9,
+         'profit_percent': '',
+         'profit_abs': '',
+         'open_time': np.datetime64('2018-10-03T00:20:00.000000000'),
+         'close_time': np.datetime64('2018-10-03T00:25:00.000000000'),
+         'open_index': 4,
+         'close_index': 4,
+         'trade_duration': '',
+         'open_rate': 20,
+         'close_rate': 10,
+         'exit_type': 'sell_signal'},
+        {'pair': 'TEST/BTC',
+         'stoploss': -0.9,
+         'profit_percent': '',
+         'profit_abs': '',
+         'open_time': np.datetime64('2018-10-03T00:20:00.000000000'),
+         'close_time': np.datetime64('2018-10-03T00:25:00.000000000'),
+         'open_index': 4,
+         'close_index': 4,
+         'trade_duration': '',
+         'open_rate': 20,
+         'close_rate': 10,
+         'exit_type': 'sell_signal'},
+        {'pair': 'TEST/BTC',
+         'stoploss': -0.9,
+         'profit_percent': '',
+         'profit_abs': '',
+         'open_time': np.datetime64('2018-10-03T00:20:00.000000000'),
+         'close_time': np.datetime64('2018-10-03T00:25:00.000000000'),
+         'open_index': 4,
+         'close_index': 4,
+         'trade_duration': '',
+         'open_rate': 20,
+         'close_rate': 10,
+         'exit_type': 'sell_signal'},
+        {'pair': 'TEST/BTC',
+         'stoploss': -0.9,
+         'profit_percent': '',
+         'profit_abs': '',
+         'open_time': np.datetime64('2018-10-03T00:20:00.000000000'),
+         'close_time': np.datetime64('2018-10-03T00:25:00.000000000'),
+         'open_index': 4,
+         'close_index': 4,
+         'trade_duration': '',
+         'open_rate': 20,
+         'close_rate': 10,
+         'exit_type': 'sell_signal'},
+
+        {'pair': 'TEST/BTC',
+         'stoploss': -0.9,
+         'profit_percent': '',
+         'profit_abs': '',
+         'open_time': np.datetime64('2018-10-03T00:30:00.000000000'),
+         'close_time': np.datetime64('2018-10-03T00:40:00.000000000'),
+         'open_index': 6,
+         'close_index': 7,
+         'trade_duration': '',
+         'open_rate': 26,
+         'close_rate': 134,
+         'exit_type': 'sell_signal'}
+    ]
+
+    trades_df = DataFrame(trades)
+    trades_df = edge._fill_calculable_fields(trades_df)
+    final = edge._process_expectancy(trades_df)
+
+    assert 'TEST/BTC' in final
+    assert final['TEST/BTC'].stoploss == -0.9
+    assert final['TEST/BTC'].nb_trades == len(trades_df) - 1
+    assert round(final['TEST/BTC'].winrate, 10) == 0.0