Repository: rotki/rotki

Issue:
Can't get Bitstamp trades
## Problem Definition

I have connrcted to Bitstamp. When I try to view trades I get the error:

> Failed query for Bitstamp: Error: The backend query task died unexpectedly: near "AND": syntax error


<!-- Provide a description of what is the current problem and why you are raising this issue.
If it's a bug please describe what was the unexpected thing that occured and what was the
expected behaviour. -->

## Logs

rotkehlchen.log is empty.

<!-- Rotki also logs debug information to the `rotkehlchen.log` file. Please attach it to the
issue as it may help us find the source of the issue faster. You can find the logs location by going
to the Help menu on the top and then choosing "Logs directory" -->


### System Description


Operating system:  Fedora Linux 32
Rotki version: 1.10.1



Discussion:
@jenkadit By default, the login is only set to critical, would it be possible for you to change the log level to debug and check the logs again?

You can change it at the login screen by pressing this button in the lower right corner.
![image](https://user-images.githubusercontent.com/2269732/102354685-35d45f00-3fab-11eb-98fd-8cfbf01c3924.png)

After that, you should log in again. As soon as you get the same error there should be a stacktrace (It starts with Traceback) in the logs that have a similar message to that one above.

Would it be possible to give us this stacktrace after removing any private information, if they exist? That would help us a lot to resolve the issue.

This seems to be the revant part from the log:

```
[16/12/2020 15:12:17 CET] DEBUG rotkehlchen.api.rest: Request successful response={"result": {"task_id": 10}, "message": ""}, status_code=200
[16/12/2020 15:12:17 CET] INFO rotkehlchen.api.server.pywsgi: 127.0.0.1 - - "GET /api/1/trades?async_query=true&location=bitstamp HTTP/1.1" 200 230 0.003946
[16/12/2020 15:12:17 CET] DEBUG rotkehlchen.chain.ethereum.graph: Querying The Graph for query ($limit: Int!, $offset: Int!, $address: Bytes!, $start_ts: BigInt!, $end_ts: BigInt!){swaps ( first: $limit, skip: $offset, where: { to: $address, timestamp_gte: $start_ts, timestamp_lte: $end_ts, } ) { transaction { swaps { id logIndex sender to timestamp pair { token0 { id decimals name symbol } token1 { id decimals name symbol } } amount0In amount0Out amount1In amount1Out } } }}
[16/12/2020 15:12:17 CET] ERROR rotkehlchen.api.rest: Greenlet for task 10 dies with exception: near "AND": syntax error.
Exception Name: <class 'pysqlcipher3.dbapi2.OperationalError'>
Exception Info: near "AND": syntax error
Traceback:
   File "src/gevent/greenlet.py", line 818, in gevent._greenlet.Greenlet.run
  File "rotkehlchen/api/rest.py", line 239, in _do_query_async
  File "rotkehlchen/api/rest.py", line 543, in _get_trades
  File "rotkehlchen/rotkehlchen.py", line 597, in query_trades
  File "rotkehlchen/rotkehlchen.py", line 665, in query_location_trades
  File "rotkehlchen/utils/interfaces.py", line 115, in wrapper
  File "rotkehlchen/exchanges/exchange.py", line 155, in query_trade_history
  File "rotkehlchen/exchanges/bitstamp.py", line 242, in query_online_trade_history
  File "rotkehlchen/db/dbhandler.py", line 2312, in get_trades

[16/12/2020 15:12:17 CET] DEBUG rotkehlchen.api.rest: Request successful response={"result": {"status": "completed", "outcome": {"result": null, "message": "The backend query task died unexpectedly: near \"AND\": syntax error"}}, "message": ""}, status_code=200
```
Thanks a lot @jenkadit. We are looking into it, will fix and the fix will be included in the next release.
Hello, I have a similar error:
bitstamp balance query Failed to query bitstamp balances: Bitstamp query responded with error status code: 403 and text: Content-Type header should not be present. 
Thanks in advance!
That seems like a different error. @vnavascues can you also look into that?

@jenkadit was the bitstamp balance query succesful for you?
Yes, I see Bitstamp balance.
@jenkadit @Yamarnez thanks for having tried it and having reported it. We have found both issues and as @LefterisJP said, the bugfixes will be included in the next release.
cool thanks!

Resolving patch:
diff --git a/rotkehlchen/db/utils.py b/rotkehlchen/db/utils.py
--- a/rotkehlchen/db/utils.py
+++ b/rotkehlchen/db/utils.py
@@ -81,27 +81,24 @@ def form_query_to_filter_timestamps(
         to_ts: Optional[Timestamp],
 ) -> Tuple[str, Union[Tuple, Tuple[Timestamp], Tuple[Timestamp, Timestamp]]]:
     """Formulates the query string and its bindings to filter for timestamps"""
-    bindings: Union[Tuple, Tuple[Timestamp], Tuple[Timestamp, Timestamp]] = ()
     got_from_ts = from_ts is not None
     got_to_ts = to_ts is not None
-    if (got_from_ts or got_to_ts):
-        if 'WHERE' not in query:
-            query += 'WHERE '
-        else:
-            query += 'AND '
+    if got_from_ts or got_to_ts:
+        query += 'WHERE' if 'WHERE' not in query else 'AND '
 
+    filters = []
+    bindings = []
     if got_from_ts:
-        query += f'{timestamp_attribute} >= ? '
-        bindings = (from_ts,)
-        if got_to_ts:
-            query += f'AND {timestamp_attribute} <= ? '
-            bindings = (from_ts, to_ts)
-    elif got_to_ts:
-        query += f'AND {timestamp_attribute} <= ? '
-        bindings = (to_ts,)
+        filters.append(f'{timestamp_attribute} >= ? ')
+        bindings.append(from_ts)
+    if got_to_ts:
+        filters.append(f'{timestamp_attribute} <= ? ')
+        bindings.append(to_ts)
 
+    query += 'AND '.join(filters)
     query += f'ORDER BY {timestamp_attribute} ASC;'
-    return query, bindings
+
+    return query, tuple(bindings)
 
 
 def deserialize_tags_from_db(val: Optional[str]) -> Optional[List[str]]:
diff --git a/rotkehlchen/exchanges/bitstamp.py b/rotkehlchen/exchanges/bitstamp.py
--- a/rotkehlchen/exchanges/bitstamp.py
+++ b/rotkehlchen/exchanges/bitstamp.py
@@ -115,7 +115,8 @@ def __init__(
         super().__init__('bitstamp', api_key, secret, database)
         self.base_uri = 'https://www.bitstamp.net/api'
         self.msg_aggregator = msg_aggregator
-        # NB: X-Auth-Signature, X-Auth-Nonce, X-Auth-Timestamp and Content-Type per request
+        # NB: X-Auth-Signature, X-Auth-Nonce, X-Auth-Timestamp & Content-Type change per request
+        # X-Auth and X-Auth-Version are constant
         self.session.headers.update({
             'X-Auth': f'BITSTAMP {self.api_key}',
             'X-Auth-Version': 'v2',
@@ -313,6 +314,8 @@ def _api_query(
         })
         if content_type:
             self.session.headers.update({'Content-Type': content_type})
+        else:
+            self.session.headers.pop('Content-Type', None)
 
         log.debug('Bitstamp API request', request_url=request_url)
         try:


Test patch:
diff --git a/rotkehlchen/tests/exchanges/test_bitstamp.py b/rotkehlchen/tests/exchanges/test_bitstamp.py
--- a/rotkehlchen/tests/exchanges/test_bitstamp.py
+++ b/rotkehlchen/tests/exchanges/test_bitstamp.py
@@ -837,3 +837,51 @@ def test_query_online_deposits_withdrawals(mock_bitstamp, start_ts, since_id):
                 end_ts=Timestamp(2),
             )
             assert mock_api_query_paginated.call_args == expected_call
+
+
+@pytest.mark.freeze_time(datetime(2020, 12, 3, 12, 0, 0))
+@pytest.mark.parametrize('bitstamp_api_key', ['123456'])
+@pytest.mark.parametrize('bitstamp_api_secret', [str.encode('abcdefg')])
+def test_api_query_request_headers_checks(mock_bitstamp):
+    """Test request headers are not polluted by previous requests
+    """
+    options = {
+        'limit': API_MAX_LIMIT,
+        'since_id': USER_TRANSACTION_MIN_SINCE_ID,
+        'sort': USER_TRANSACTION_SORTING_MODE,
+        'offset': 0,
+    }
+    uuid = MagicMock()
+    uuid.uuid4.return_value = 'hijklm'
+
+    session = mock_bitstamp.session
+    with patch('rotkehlchen.exchanges.bitstamp.uuid', new=uuid):
+        mock_bitstamp._api_query(endpoint='balance')
+        assert session.headers['X-Auth'] == 'BITSTAMP 123456'
+        assert session.headers['X-Auth-Version'] == 'v2'
+        assert session.headers['X-Auth-Signature'] == (
+            'eb84d115027532cba9ebab8c692c488284c54551ab4601aa9ce6280187dc9c86'
+        )
+        assert session.headers['X-Auth-Nonce'] == 'hijklm'
+        assert session.headers['X-Auth-Timestamp'] == '1606996800000'
+        assert 'Content-Type' not in session.headers
+
+        mock_bitstamp._api_query(endpoint='balance', options=options.copy())
+        assert session.headers['X-Auth'] == 'BITSTAMP 123456'
+        assert session.headers['X-Auth-Version'] == 'v2'
+        assert session.headers['X-Auth-Signature'] == (
+            '29728913d776144f0c8d522a58e77bb6c4492b25dbf7b3ebd41c4eb64c28cf0c'
+        )
+        assert session.headers['X-Auth-Nonce'] == 'hijklm'
+        assert session.headers['X-Auth-Timestamp'] == '1606996800000'
+        assert session.headers['Content-Type'] == 'application/x-www-form-urlencoded'
+
+        mock_bitstamp._api_query(endpoint='balance')
+        assert session.headers['X-Auth'] == 'BITSTAMP 123456'
+        assert session.headers['X-Auth-Version'] == 'v2'
+        assert session.headers['X-Auth-Signature'] == (
+            'eb84d115027532cba9ebab8c692c488284c54551ab4601aa9ce6280187dc9c86'
+        )
+        assert session.headers['X-Auth-Nonce'] == 'hijklm'
+        assert session.headers['X-Auth-Timestamp'] == '1606996800000'
+        assert 'Content-Type' not in session.headers
diff --git a/rotkehlchen/tests/fixtures/exchanges/bitstamp.py b/rotkehlchen/tests/fixtures/exchanges/bitstamp.py
--- a/rotkehlchen/tests/fixtures/exchanges/bitstamp.py
+++ b/rotkehlchen/tests/fixtures/exchanges/bitstamp.py
@@ -1,6 +1,17 @@
 import pytest
 
 from rotkehlchen.tests.utils.exchanges import create_test_bitstamp
+from rotkehlchen.tests.utils.factories import make_api_key, make_api_secret
+
+
+@pytest.fixture
+def bitstamp_api_key():
+    return make_api_key()
+
+
+@pytest.fixture
+def bitstamp_api_secret():
+    return make_api_secret()
 
 
 @pytest.fixture
@@ -8,8 +19,12 @@ def mock_bitstamp(
         database,
         inquirer,  # pylint: disable=unused-argument
         function_scope_messages_aggregator,
+        bitstamp_api_key,
+        bitstamp_api_secret,
 ):
     return create_test_bitstamp(
         database=database,
         msg_aggregator=function_scope_messages_aggregator,
+        api_key=bitstamp_api_key,
+        secret=bitstamp_api_secret,
     )
diff --git a/rotkehlchen/tests/utils/exchanges.py b/rotkehlchen/tests/utils/exchanges.py
--- a/rotkehlchen/tests/utils/exchanges.py
+++ b/rotkehlchen/tests/utils/exchanges.py
@@ -309,10 +309,17 @@ def create_test_bitmex(
 def create_test_bitstamp(
         database,
         msg_aggregator,
+        api_key=None,
+        secret=None,
 ) -> Bitstamp:
+    if api_key is None:
+        api_key = make_api_key()
+    if secret is None:
+        secret = make_api_secret()
+
     return Bitstamp(
-        api_key=make_api_key(),
-        secret=make_api_secret(),
+        api_key=api_key,
+        secret=secret,
         database=database,
         msg_aggregator=msg_aggregator,
     )