Repository: freqtrade/freqtrade

Issue:
No pairs file found with path "user_data/data/binance/pairs.json", --refresh-pairs-cached downloads a different list

  * Operating system: Ubuntu 18.04 
  * Python Version: _____ 3.6.8
  * CCXT version: _____ ccxt==1.18.1043
  * Branch: Develop
  * Last Commit ID: _____ 8c1efec43a71b4e28312010fb8691bfe8badffd7
  
My pairs.json is found in user_data/data/binance/pairs.json but the following error appears:

2019-08-13 17:38:36,164 - freqtrade.configuration.configuration - INFO - Using data directory: user_data/data/binance ...
2019-08-13 17:38:36,164 - download_backtest_data - INFO - Reading pairs file "user_data/data/binance/pairs.json".
No pairs file found with path "user_data/data/binance/pairs.json".



Discussion:
Only possible with the download_backtest_data script, but the -r command did not work in the command line.
But -r is not fetching the same list as the user_data/data/pairs.json, when using freqtrade hyepropts for example. It used to work a couple of days ago.

It's not quite clear what you're trying to do ... 

first of all: 
The download-script does not have a `-r` option ... ? 
No other option having the `-r` option accepts the pairs-file parameter ...

if you get `2019-08-13 17:38:36,164 - download_backtest_data - INFO - Reading pairs file "user_data/data/binance/pairs.json".
No pairs file found with path "user_data/data/binance/pairs.json".` - then the download-script will fallback to the whitelist in the configuration.

Can you show the output of the `cat pairs.json`? (i would assume it has a syntax-error - preventing it from loading correctly).
Ok, I have downloaded my pairs.json list with the download-script.
I then go to hyperopts, and use the option -r, but the downloaded list is different from the script and therefore different from the pairs.json. Even when I am using the last 5 days it does not download the same list of pairs. I just selected all binance pairs, except BCHABC/BTC and YOYO/BTC.
Oh, it is a different list in the config file and it seems that is the one it downloads. But even then I had an error, now it seems to be gone. It had something to do with missing information.


`pairs.json` is ONLY used by the download script. it is not transferred to the configuration, nor is it used by backtesting / hyperopt.

also - if you use the download script to get data - please **don't** use `-r` - it'll overwrite the data (but based on the whitelist in the configuration), so running the download-script was pointless.

As mentioned before - i think the `-r` option is confusing and misleading - i'm still in favor of removing it and point users to the download-script...

I did not get what "but the -r command did not work in the command line" can mean. '-r' is an option for the optimization freqtrade subcommands. Can you explain the particular sequence of actions that can be reproduced?

Also having some issues to download pairs data using pairs.json file. I placed the file at ``freqtrade/userdata/data/binance`` but after running the command ``freqtrade download-data --exchange binance`` I get the below error message.

![error](https://user-images.githubusercontent.com/46167400/63391646-976dc480-c389-11e9-8726-3111bc9c401d.JPG)




Resolving patch:
diff --git a/freqtrade/configuration/configuration.py b/freqtrade/configuration/configuration.py
--- a/freqtrade/configuration/configuration.py
+++ b/freqtrade/configuration/configuration.py
@@ -346,10 +346,9 @@ def _resolve_pairs_list(self, config: Dict[str, Any]) -> None:
             # or if pairs file is specified explicitely
             if not pairs_file.exists():
                 raise OperationalException(f'No pairs file found with path "{pairs_file}".')
-
-            config['pairs'] = json_load(pairs_file)
-
-            config['pairs'].sort()
+            with pairs_file.open('r') as f:
+                config['pairs'] = json_load(f)
+                config['pairs'].sort()
             return
 
         if "config" in self.args and self.args.config:
@@ -357,7 +356,10 @@ def _resolve_pairs_list(self, config: Dict[str, Any]) -> None:
             config['pairs'] = config.get('exchange', {}).get('pair_whitelist')
         else:
             # Fall back to /dl_path/pairs.json
-            pairs_file = Path(config['datadir']) / "pairs.json"
+            pairs_file = Path(config['datadir']) / config['exchange']['name'].lower() / "pairs.json"
+            print(config['datadir'])
             if pairs_file.exists():
-                config['pairs'] = json_load(pairs_file)
-            config['pairs'].sort()
+                with pairs_file.open('r') as f:
+                    config['pairs'] = json_load(f)
+                if 'pairs' in config:
+                    config['pairs'].sort()
diff --git a/freqtrade/misc.py b/freqtrade/misc.py
--- a/freqtrade/misc.py
+++ b/freqtrade/misc.py
@@ -6,6 +6,7 @@
 import re
 from datetime import datetime
 from pathlib import Path
+from typing.io import IO
 
 import numpy as np
 import rapidjson
@@ -60,7 +61,7 @@ def file_dump_json(filename: Path, data, is_zip=False) -> None:
     logger.debug(f'done json to "{filename}"')
 
 
-def json_load(datafile):
+def json_load(datafile: IO):
     """
     load data with rapidjson
     Use this to have a consistent experience,


Test patch:
diff --git a/freqtrade/tests/test_configuration.py b/freqtrade/tests/test_configuration.py
--- a/freqtrade/tests/test_configuration.py
+++ b/freqtrade/tests/test_configuration.py
@@ -770,6 +770,7 @@ def test_pairlist_resolving_with_config_pl(mocker, default_conf):
     load_mock = mocker.patch("freqtrade.configuration.configuration.json_load",
                              MagicMock(return_value=['XRP/BTC', 'ETH/BTC']))
     mocker.patch.object(Path, "exists", MagicMock(return_value=True))
+    mocker.patch.object(Path, "open", MagicMock(return_value=MagicMock()))
 
     arglist = [
         '--config', 'config.json',
@@ -808,6 +809,7 @@ def test_pairlist_resolving_with_config_pl_not_exists(mocker, default_conf):
 
 def test_pairlist_resolving_fallback(mocker):
     mocker.patch.object(Path, "exists", MagicMock(return_value=True))
+    mocker.patch.object(Path, "open", MagicMock(return_value=MagicMock()))
     mocker.patch("freqtrade.configuration.configuration.json_load",
                  MagicMock(return_value=['XRP/BTC', 'ETH/BTC']))
     arglist = [