Repository: freqtrade/freqtrade

Issue:
binanceus attempts to download from binance.com
## Describe your environment

  * Operating system: debian 12.4
  * Python Version: Python 3.11.2
  * CCXT version: ccxt==4.1.66
  * Freqtrade Version: 2023.12-dev-cfd5af67c & 2023.11

## Describe the problem:

When using binanceus it downloads data for binanceus for 3m 5m 15m 1h 4h 8h fine but attempts to download 8h data from binance.com after downloading from binanceus

### Steps to reproduce:

  1. fresh install either stable or develop
  2. run freqtrade trade --config user_data/config_ffreqtrade trade --config user_data/config_freqai.json --strategy FreqaiExampleStrategy --freqaimodel XGBoostRegressor --strategy-path user_data/strategies
  3. Error is output when attempting to download 8h from binance.com
  
### Observed Results:

  data is downloaded from binance.com resulting in error

### Relevant code exceptions or logs
 ```
2023-12-13 22:14:42,553 - freqtrade.configuration.config_validation - INFO - Validating configuration ...
2023-12-13 22:14:42,557 - freqtrade.exchange.exchange - INFO - Instance is running with dry_run enabled
2023-12-13 22:14:42,557 - freqtrade.exchange.exchange - INFO - Using CCXT 4.1.66
2023-12-13 22:14:42,558 - freqtrade.exchange.exchange - INFO - Applying additional ccxt config: {'options': {'defaultType': 'swap'}, 'enableRateLimit': True}
2023-12-13 22:14:42,571 - freqtrade.exchange.exchange - INFO - Applying additional ccxt config: {'options': {'defaultType': 'swap'}, 'enableRateLimit': True, 'rateLimit': 1000}
2023-12-13 22:14:42,584 - freqtrade.exchange.exchange - INFO - Using Exchange "Binance US"
2023-12-13 22:14:43,979 - freqtrade.resolvers.exchange_resolver - INFO - Using resolved exchange 'Binance'...
2023-12-13 22:14:44,083 - freqtrade.wallets - INFO - Wallets synced.
2023-12-13 22:14:44,095 - freqtrade.resolvers.iresolver - INFO - Using resolved pairlist StaticPairList from '/nfs/build/freqtrade/freqtrade/plugins/pairlist/StaticPairList.py'...
2023-12-13 22:14:44,573 - freqtrade.resolvers.iresolver - INFO - Using resolved pairlist StaticPairList from '/nfs/build/freqtrade/freqtrade/plugins/pairlist/StaticPairList.py'...
2023-12-13 22:14:44,577 - freqtrade.plugins.pairlistmanager - INFO - Whitelist with 0 pairs: []
2023-12-13 22:14:48,266 - freqtrade.resolvers.iresolver - INFO - Using resolved freqaimodel XGBoostRegressor from '/nfs/build/freqtrade/freqtrade/freqai/prediction_models/XGBoostRegressor.py'...
2023-12-13 22:14:48,266 - freqtrade.freqai.freqai_interface - INFO - Backtesting module configured to save all models.
2023-12-13 22:14:48,267 - freqtrade.freqai.data_drawer - INFO - Could not find existing datadrawer, starting from scratch
2023-12-13 22:14:48,267 - freqtrade.freqai.data_drawer - INFO - Could not find existing historic_predictions, starting from scratch
2023-12-13 22:14:48,267 - freqtrade.freqai.data_drawer - INFO - Could not find existing metric tracker, starting from scratch
2023-12-13 22:14:48,267 - freqtrade.freqai.freqai_interface - INFO - Set fresh train queue from whitelist. Queue: ['.*/USDT']
2023-12-13 22:14:48,268 - freqtrade.strategy.interface - INFO - Downloading all training data for all pairs in whitelist and corr_pairlist, this may take a while if the data is not already on disk.
2023-12-13 22:14:48,268 - freqtrade.freqai.utils - INFO - FreqAI auto-downloader using 120 startup candles.
2023-12-13 22:14:48,327 - freqtrade.data.history.history_utils - INFO - (1/160) - Download history data for "BTC/USDT", 3m, futures and store in /nfs/build/freqtrade/user_data/data/binanceus. From 2023-12-13T22:00:00 to 2023-12-13T22:14:48
2023-12-13 22:14:52,389 - freqtrade.exchange.exchange - INFO - Downloaded data for BTC/USDT with length 5.
2023-12-13 22:14:52,563 - freqtrade.data.history.history_utils - INFO - (1/160) - Download history data for "BTC/USDT", 5m, futures and store in /nfs/build/freqtrade/user_data/data/binanceus. From 2023-12-13T21:55:00 to 2023-12-13T22:14:48
2023-12-13 22:14:53,092 - freqtrade.exchange.exchange - INFO - Downloaded data for BTC/USDT with length 4.
2023-12-13 22:14:53,205 - freqtrade.data.history.history_utils - INFO - (1/160) - Download history data for "BTC/USDT", 15m, futures and store in /nfs/build/freqtrade/user_data/data/binanceus. From 2023-10-20T23:14:48 to 2023-12-13T22:14:48
2023-12-13 22:14:56,211 - freqtrade.exchange.exchange - INFO - Downloaded data for BTC/USDT with length 5184.
2023-12-13 22:14:56,288 - freqtrade.data.history.history_utils - INFO - (1/160) - Download history data for "BTC/USDT", 1h, futures and store in /nfs/build/freqtrade/user_data/data/binanceus. From 2023-10-20T23:14:48 to 2023-12-13T22:14:48
2023-12-13 22:14:57,677 - freqtrade.exchange.exchange - INFO - Downloaded data for BTC/USDT with length 1296.
2023-12-13 22:14:57,737 - freqtrade.data.history.history_utils - INFO - (1/160) - Download history data for "BTC/USDT", 4h, futures and store in /nfs/build/freqtrade/user_data/data/binanceus. From 2023-10-20T23:14:48 to 2023-12-13T22:14:48
2023-12-13 22:14:58,735 - freqtrade.exchange.exchange - INFO - Downloaded data for BTC/USDT with length 324.
2023-12-13 22:14:58,786 - freqtrade.data.history.history_utils - INFO - (1/160) - Download history data for "BTC/USDT", 8h, futures and store in /nfs/build/freqtrade/user_data/data/binanceus. From 2023-10-20T23:14:48 to 2023-12-13T22:14:48
2023-12-13 22:14:59,761 - freqtrade.exchange.exchange - INFO - Downloaded data for BTC/USDT with length 162.
2023-12-13 22:14:59,788 - freqtrade.data.history.history_utils - INFO - (1/160) - Download history data for "BTC/USDT", 8h, funding_rate and store in /nfs/build/freqtrade/user_data/data/binanceus. From 2023-10-20T23:14:48 to 2023-12-13T22:14:48
2023-12-13 22:15:01,016 - freqtrade.exchange.common - WARNING - _async_get_candle_history() returned exception: "Could not fetch historical candle (OHLCV) data for pair BTC/USDT due to ExchangeNotAvailable. Message: binanceus GET https://fapi.binance.com/fapi/v1/fundingRate?symbol=BTCUSDT&startTime=0&limit=1000 451  {
  "code": 0,
  "msg": "Service unavailable from a restricted location according to 'b. Eligibility' in https://www.binance.com/en/terms. Please contact customer service if you believe you received this message in error."
}". Retrying still for 4 times.
2023-12-13 22:15:02,201 - freqtrade.exchange.common - WARNING - _async_get_candle_history() returned exception: "Could not fetch historical candle (OHLCV) data for pair BTC/USDT due to ExchangeNotAvailable. Message: binanceus GET https://fapi.binance.com/fapi/v1/fundingRate?symbol=BTCUSDT&startTime=0&limit=1000 451  {
  "code": 0,
  "msg": "Service unavailable from a restricted location according to 'b. Eligibility' in https://www.binance.com/en/terms. Please contact customer service if you believe you received this message in error."
}". Retrying still for 3 times.
^C2023-12-13 22:15:03,280 - freqtrade.commands.trade_commands - INFO - SIGINT received, aborting ...

  ```



Discussion:
There's several problems here... none of which are actual software problems, but rather configuration problems.

* binance.us does not have futures markets, only spot markets (at least as far as i can see from europe)
* You seem to be aware of this, as the configured pairs (e.g. BTC/USDT) are spot markets (futures markets would be BTC/USDT:USDT).
* you've got freqtrade configured to use futures markets, which attempts to download funding fees (something that only exists on futures markets) - which (as futures are not available on binance.us) is an endpoint that most likely doesn't even exist on binance.us.

Initially, i thought it's a ccxt problem (hence the temporary label) where they route the request wrongly - but in reality, it's a simple configuration problem.

In your configuration - remove the setting `"trading_mode": "futures"` (or set it to spot) - and you should be able to use binance.us markets.

Now why this is even possible i'll have to look at - as in reality - these markets should be "thrown out" when trying to download futures data, but i suspect it'll come from some odd response in the binance.us exchange response.

---

Please be aware however that in my experience, binance.us markets are quite quickly markets with a very light ordebook - which usually means that markets can be pushed around by "not so large" players quite easily.



Resolving patch:
diff --git a/freqtrade/freqai/utils.py b/freqtrade/freqai/utils.py
--- a/freqtrade/freqai/utils.py
+++ b/freqtrade/freqai/utils.py
@@ -13,7 +13,6 @@
 from freqtrade.data.history.history_utils import refresh_backtest_ohlcv_data
 from freqtrade.exceptions import OperationalException
 from freqtrade.exchange import timeframe_to_seconds
-from freqtrade.exchange.exchange import market_is_active
 from freqtrade.freqai.data_drawer import FreqaiDataDrawer
 from freqtrade.freqai.data_kitchen import FreqaiDataKitchen
 from freqtrade.plugins.pairlist.pairlist_helpers import dynamic_expand_pairlist
@@ -33,8 +32,11 @@ def download_all_data_for_training(dp: DataProvider, config: Config) -> None:
 
     if dp._exchange is None:
         raise OperationalException('No exchange object found.')
-    markets = [p for p, m in dp._exchange.markets.items() if market_is_active(m)
-               or config.get('include_inactive')]
+    markets = [
+        p for p in dp._exchange.get_markets(
+            tradable_only=True, active_only=not config.get('include_inactive')
+            ).keys()
+    ]
 
     all_pairs = dynamic_expand_pairlist(config, markets)
 


Test patch: