Repository: ccxt/ccxt
Pull request: fix(coinex): watchTickers - new watchTickers implementation
There are two issues with the current watchTickers implementation

- it can hit amaximum recursion depth error (Fix #18015)
- if a user calls watchTickers (['BTC/USDT']) and watchTickers (['BTC/USDT', ETH/USDT']) and a new BTC/USDT ticker is received only the first ticker will resolve, when I believe on userland it would make sense for both to resolve. (It's been a while since I've tested this but I believe it is still an issue)

This PR proposes the following fix:
1. Add the symbols to the messageHash
2. When a message is received find all messageHashes that contain the symbol to resolve
3. Build array of tickers for that message hash to resolve.
4. Resolve each messageHash
5. In watchTickers check if to return only new updates or all tickers filtered by that symbol

Alternative solution. Use a new cache object to handle this like the one proposed for `watchPositions` in this PR #15622

If we decide this as a good solution it should then be applied to the rest of `watchTickers` implementations.



Problem statement:
CoinEx watchTicker(): maximum recursion depth exceeded while calling a Python object
### Operating System

Ubuntu 20.04.5 LTS

### Programming Languages

python

### CCXT Version

3.1.6

### Description

The problem may occur a few minutes or a few hours after starting the program:

```
2023-05-25 11:37:39,543 ERROR maximum recursion depth exceeded while calling a Python object: Traceback (most recent call last):
  File "main.py", line 366, in watch_ticker
    await exchange.watch_ticker(symbol)
  File "/home/user/.local/lib/python3.8/site-packages/ccxt/pro/coinex.py", line 375, in watch_ticker
    return await self.watch_tickers([symbol], params)
  File "/home/user/.local/lib/python3.8/site-packages/ccxt/pro/coinex.py", line 405, in watch_tickers
    return await self.watch_tickers(symbols, params)
  File "/home/user/.local/lib/python3.8/site-packages/ccxt/pro/coinex.py", line 405, in watch_tickers
    return await self.watch_tickers(symbols, params)
  File "/home/user/.local/lib/python3.8/site-packages/ccxt/pro/coinex.py", line 405, in watch_tickers
    return await self.watch_tickers(symbols, params)
  [Previous line repeated 981 more times]
  File "/home/user/.local/lib/python3.8/site-packages/ccxt/pro/coinex.py", line 388, in watch_tickers
    type, params = self.handle_market_type_and_params('watchTickers', None, params)
  File "/home/user/.local/lib/python3.8/site-packages/ccxt/async_support/base/exchange.py", line 1925, in handle_market_type_and_params
    defaultType = self.safe_string_2(self.options, 'defaultType', 'type', 'spot')
  File "/home/user/.local/lib/python3.8/site-packages/ccxt/base/exchange.py", line 706, in safe_string_2
    return Exchange.safe_either(Exchange.safe_string, dictionary, key1, key2, default_value)
  File "/home/user/.local/lib/python3.8/site-packages/ccxt/base/exchange.py", line 818, in safe_either
    value = method(dictionary, key1)
  File "/home/user/.local/lib/python3.8/site-packages/ccxt/base/exchange.py", line 645, in safe_string
    return str(dictionary[key]) if Exchange.key_exists(dictionary, key) else default_value
  File "/home/user/.local/lib/python3.8/site-packages/ccxt/base/exchange.py", line 623, in key_exists
    if hasattr(dictionary, 'getitem') and not isinstance(dictionary, str):
RecursionError: maximum recursion depth exceeded while calling a Python object

2023-05-25 11:37:39,602 ERROR Task exception was never retrieved
future: <Task finished name='Task-5' coro=<exchange_loop() done, defined at main.py:305> exception=SystemExit(None)>
Traceback (most recent call last):
  File "main.py", line 366, in watch_ticker
    await exchange.watch_ticker(symbol)
  File "/home/user/.local/lib/python3.8/site-packages/ccxt/pro/coinex.py", line 375, in watch_ticker
    return await self.watch_tickers([symbol], params)
  File "/home/user/.local/lib/python3.8/site-packages/ccxt/pro/coinex.py", line 405, in watch_tickers
    return await self.watch_tickers(symbols, params)
  File "/home/user/.local/lib/python3.8/site-packages/ccxt/pro/coinex.py", line 405, in watch_tickers
    return await self.watch_tickers(symbols, params)
  File "/home/user/.local/lib/python3.8/site-packages/ccxt/pro/coinex.py", line 405, in watch_tickers
    return await self.watch_tickers(symbols, params)
  [Previous line repeated 981 more times]
  File "/home/user/.local/lib/python3.8/site-packages/ccxt/pro/coinex.py", line 388, in watch_tickers
    type, params = self.handle_market_type_and_params('watchTickers', None, params)
  File "/home/user/.local/lib/python3.8/site-packages/ccxt/async_support/base/exchange.py", line 1925, in handle_market_type_and_params
    defaultType = self.safe_string_2(self.options, 'defaultType', 'type', 'spot')
  File "/home/user/.local/lib/python3.8/site-packages/ccxt/base/exchange.py", line 706, in safe_string_2
    return Exchange.safe_either(Exchange.safe_string, dictionary, key1, key2, default_value)
  File "/home/user/.local/lib/python3.8/site-packages/ccxt/base/exchange.py", line 818, in safe_either
    value = method(dictionary, key1)
  File "/home/user/.local/lib/python3.8/site-packages/ccxt/base/exchange.py", line 645, in safe_string
    return str(dictionary[key]) if Exchange.key_exists(dictionary, key) else default_value
  File "/home/user/.local/lib/python3.8/site-packages/ccxt/base/exchange.py", line 623, in key_exists
    if hasattr(dictionary, 'getitem') and not isinstance(dictionary, str):
RecursionError: maximum recursion depth exceeded while calling a Python object

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3.8/asyncio/runners.py", line 47, in run
    _cancel_all_tasks(loop)
  File "/usr/lib/python3.8/asyncio/runners.py", line 62, in _cancel_all_tasks
    loop.run_until_complete(
  File "/usr/lib/python3.8/asyncio/base_events.py", line 603, in run_until_complete
    self.run_forever()
  File "/usr/lib/python3.8/asyncio/base_events.py", line 570, in run_forever
    self._run_once()
  File "/usr/lib/python3.8/asyncio/base_events.py", line 1859, in _run_once
    handle._run()
  File "/usr/lib/python3.8/asyncio/events.py", line 81, in _run
    self._context.run(self._callback, *self._args)
  File "main.py", line 328, in exchange_loop
    await asyncio.gather(watch_balance(exchange), *tickers_loops, *loops)
  File "/usr/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/lib/python3.8/asyncio/base_events.py", line 603, in run_until_complete
    self.run_forever()
  File "/usr/lib/python3.8/asyncio/base_events.py", line 570, in run_forever
    self._run_once()
  File "/usr/lib/python3.8/asyncio/base_events.py", line 1859, in _run_once
    handle._run()
  File "/usr/lib/python3.8/asyncio/events.py", line 81, in _run
    self._context.run(self._callback, *self._args)
  File "main.py", line 375, in watch_ticker
    exit()
  File "/usr/lib/python3.8/_sitebuiltins.py", line 26, in call
    raise SystemExit(code)
SystemExit: None
```

### Code

```
     exchange = getattr(ccxt.pro, 'coinex')()
     symbol = 'NEXA/USDT'
   
   while True:
        try:
            await exchange.watch_ticker(symbol)

        except NetworkError as e:
            logging.error(f'Network error: {exchange} watch ticker {symbol}: {str(e)}: %s', traceback.format_exc())

        except Exception as e:
            logging.error(f'{exchange} watch ticker {symbol}: {str(e)}: %s', traceback.format_exc())

            exit()

```

Hints from issue discussion:

Commit message: new updates fixes

Patch to review:
diff --git a/ts/src/base/Exchange.ts b/ts/src/base/Exchange.ts
index 9d4cace870260..54ca210a588d4 100644
--- a/ts/src/base/Exchange.ts
+++ b/ts/src/base/Exchange.ts
@@ -1358,6 +1358,18 @@ export default class Exchange {
     // ------------------------------------------------------------------------
     // METHODS BELOW THIS LINE ARE TRANSPILED FROM JAVASCRIPT TO PYTHON AND PHP
 
+    findMessageHashes (client, element: string): string[] {
+        const result = [];
+        const messageHashes = Object.keys (client.futures);
+        for (let i = 0; i < messageHashes.length; i++) {
+            const messageHash = messageHashes[i];
+            if (messageHash.indexOf (element) >= 0) {
+                result.push (messageHash);
+            }
+        }
+        return result;
+    }
+
     filterByLimit (array: object[], limit: Int = undefined, key: IndexType = 'timestamp'): any {
         if (this.valueIsDefined (limit)) {
             const arrayLength = array.length;
diff --git a/ts/src/pro/coinex.ts b/ts/src/pro/coinex.ts
index 0b9a0e0a710ee..30ea76fd3f4cd 100644
--- a/ts/src/pro/coinex.ts
+++ b/ts/src/pro/coinex.ts
@@ -141,10 +141,21 @@ export default class coinex extends coinexRest {
             const symbol = this.safeSymbol (marketId, undefined, undefined, defaultType);
             const market = this.safeMarket (marketId, undefined, undefined, defaultType);
             const parsedTicker = this.parseWSTicker (rawTicker, market);
-            const messageHash = 'ticker:' + symbol;
             this.tickers[symbol] = parsedTicker;
             newTickers.push (parsedTicker);
-            client.resolve (parsedTicker, messageHash);
+        }
+        const messageHashes = this.findMessageHashes (client, 'tickers:');
+        for (let i = 0; i < messageHashes.length; i++) {
+            const messageHash = messageHashes[i];
+            const parts = messageHash.split (':');
+            const symbolsString = parts[1];
+            const symbols = symbolsString.split (',');
+            const tickers = this.filterByArray (newTickers, 'symbol', symbols);
+            const tickersSymbols = Object.keys (tickers);
+            const numTickers = tickersSymbols.length;
+            if (numTickers > 0) {
+                client.resolve (tickers, messageHash);
+            }
         }
         client.resolve (newTickers, 'tickers');
     }
@@ -403,24 +414,21 @@ export default class coinex extends coinexRest {
         let type = undefined;
         [ type, params ] = this.handleMarketTypeAndParams ('watchTickers', undefined, params);
         const url = this.urls['api']['ws'][type];
-        const messageHash = 'tickers';
+        let messageHash = 'tickers';
+        if (symbols !== undefined) {
+            messageHash = 'tickers:' + symbols.join (',');
+        }
         const subscribe = {
             'method': 'state.subscribe',
             'id': this.requestId (),
             'params': [],
         };
         const request = this.deepExtend (subscribe, params);
-        const tickers = await this.watch (url, messageHash, request, messageHash);
-        const result = this.filterByArray (tickers, 'symbol', symbols);
-        const keys = Object.keys (result);
-        const resultLength = keys.length;
-        if (resultLength > 0) {
-            if (this.newUpdates) {
-                return result;
-            }
-            return this.filterByArray (this.tickers, 'symbol', symbols);
+        const newTickers = await this.watch (url, messageHash, request, messageHash);
+        if (this.newUpdates) {
+            return newTickers;
         }
-        return await this.watchTickers (symbols, params);
+        return this.filterByArray (this.tickers, 'symbol', symbols);
     }
 
     async watchTrades (symbol: string, since: Int = undefined, limit: Int = undefined, params = {}) {


Code review comments:
diff_hunk: @@ -141,10 +141,21 @@ export default class coinex extends coinexRest {
             const symbol = this.safeSymbol (marketId, undefined, undefined, defaultType);
             const market = this.safeMarket (marketId, undefined, undefined, defaultType);
             const parsedTicker = this.parseWSTicker (rawTicker, market);
-            const messageHash = 'ticker:' + symbol;
             this.tickers[symbol] = parsedTicker;
             newTickers.push (parsedTicker);
-            client.resolve (parsedTicker, messageHash);
+        }
+        const messageHashes = this.findMessageHashes (client, 'tickers:');
+        for (let i = 0; i < messageHashes.length; i++) {
+            const messageHash = messageHashes[i];
+            const parts = messageHash.split (':');
path: ts/src/pro/coinex.ts
text: @user1:
@author hum we can't use `:` here safely, what if you have `messageHash = 'tickers:BTC/USDT,BTC/USDT:USDT'` 

Then parts would be `['tickers', 'BTC/USDT,BTC/USDT', 'USDT']`

@author:
good catch.. I replaced it for '::'