Repository: rotki/rotki

Issue:
Blockchain balances don't load when connected to local ethereum node
## Problem Definition

The problem happens when running a local ethereum node using the command `geth --rpc --syncmode=fast`.
After signing into rotkehlchen, my blockchain assets do not load on the dashboard and show as all 0's in the user settings.
I would expect that since my binance assets load on the dashboard that my blockchain assets would also load. Aren't the balances supposed to be saved?

```
19/09/2018 21:15:53 PDT -- DEBUG:rotkehlchen.rotkehlchen:Main loop start
19/09/2018 21:15:53 PDT -- DEBUG:rotkehlchen.rotkehlchen:Maybe upload to server
19/09/2018 21:15:53 PDT -- DEBUG:rotkehlchen.rotkehlchen:Main loop end
19/09/2018 21:15:59 PDT -- DEBUG:rotkehlchen.server:NEW TASK query_exchange_balances (kwargs:{'name': 'binance'}) with ID: 0
19/09/2018 21:15:59 PDT -- DEBUG:rotkehlchen.server:NEW TASK query_blockchain_balances (kwargs:{}) with ID: 1
19/09/2018 21:16:00 PDT -- DEBUG:rotkehlchen.server:Querying task result with task id 0
19/09/2018 21:16:00 PDT -- DEBUG:rotkehlchen.server:Querying task result with task id 1
19/09/2018 21:16:02 PDT -- DEBUG:rotkehlchen.server:Querying task result with task id 0
19/09/2018 21:16:02 PDT -- DEBUG:rotkehlchen.server:Querying task result with task id 1
19/09/2018 21:16:04 PDT -- DEBUG:rotkehlchen.server:Querying task result with task id 0
19/09/2018 21:16:04 PDT -- DEBUG:rotkehlchen.server:Querying task result with task id 1
19/09/2018 21:16:04 PDT -- DEBUG:rotkehlchen.server:Found response for task 1
19/09/2018 21:16:06 PDT -- DEBUG:rotkehlchen.server:Querying task result with task id 0
19/09/2018 21:16:08 PDT -- DEBUG:rotkehlchen.server:Querying task result with task id 0
19/09/2018 21:16:10 PDT -- DEBUG:rotkehlchen.server:Querying task result with task id 0
19/09/2018 21:16:10 PDT -- DEBUG:rotkehlchen.server:Found response for task 0
19/09/2018 21:16:10 PDT -- DEBUG:rotkehlchen.server:NEW TASK query_balances (kwargs:{}) with ID: 2
19/09/2018 21:16:12 PDT -- DEBUG:rotkehlchen.server:Querying task result with task id 2
19/09/2018 21:16:12 PDT -- DEBUG:rotkehlchen.server:Found response for task 2
19/09/2018 21:16:53 PDT -- DEBUG:rotkehlchen.rotkehlchen:Main loop start
19/09/2018 21:16:53 PDT -- DEBUG:rotkehlchen.rotkehlchen:Maybe upload to server
19/09/2018 21:16:53 PDT -- DEBUG:rotkehlchen.rotkehlchen:Main loop end
19/09/2018 21:17:53 PDT -- DEBUG:rotkehlchen.rotkehlchen:Main loop start
19/09/2018 21:17:53 PDT -- DEBUG:rotkehlchen.rotkehlchen:Maybe upload to server
19/09/2018 21:17:53 PDT -- DEBUG:rotkehlchen.rotkehlchen:Main loop end
19/09/2018 21:18:25 PDT -- DEBUG:rotkehlchen.server:NEW TASK query_blockchain_balances (kwargs:{}) with ID: 3
19/09/2018 21:18:26 PDT -- DEBUG:rotkehlchen.server:Querying task result with task id 3
19/09/2018 21:18:26 PDT -- DEBUG:rotkehlchen.server:Found response for task 3
19/09/2018 21:18:53 PDT -- DEBUG:rotkehlchen.rotkehlchen:Main loop start
19/09/2018 21:18:53 PDT -- DEBUG:rotkehlchen.rotkehlchen:Maybe upload to server
19/09/2018 21:18:53 PDT -- DEBUG:rotkehlchen.rotkehlchen:Main loop end
19/09/2018 21:19:53 PDT -- DEBUG:rotkehlchen.rotkehlchen:Main loop start
19/09/2018 21:19:53 PDT -- DEBUG:rotkehlchen.rotkehlchen:Maybe upload to server
19/09/2018 21:19:53 PDT -- DEBUG:rotkehlchen.rotkehlchen:Main loop end
19/09/2018 21:20:53 PDT -- DEBUG:rotkehlchen.rotkehlchen:Main loop start
19/09/2018 21:20:53 PDT -- DEBUG:rotkehlchen.rotkehlchen:Maybe upload to server
19/09/2018 21:20:53 PDT -- DEBUG:rotkehlchen.rotkehlchen:Main loop end
19/09/2018 21:21:53 PDT -- DEBUG:rotkehlchen.rotkehlchen:Main loop start
19/09/2018 21:21:53 PDT -- DEBUG:rotkehlchen.rotkehlchen:Maybe upload to server
19/09/2018 21:21:53 PDT -- DEBUG:rotkehlchen.rotkehlchen:Main loop end
19/09/2018 21:22:53 PDT -- DEBUG:rotkehlchen.rotkehlchen:Main loop start
19/09/2018 21:22:53 PDT -- DEBUG:rotkehlchen.rotkehlchen:Maybe upload to server
19/09/2018 21:22:53 PDT -- DEBUG:rotkehlchen.rotkehlchen:Main loop end
19/09/2018 21:23:53 PDT -- DEBUG:rotkehlchen.rotkehlchen:Main loop start
19/09/2018 21:23:53 PDT -- DEBUG:rotkehlchen.rotkehlchen:Maybe upload to server
19/09/2018 21:23:53 PDT -- DEBUG:rotkehlchen.rotkehlchen:Main loop end
19/09/2018 21:24:53 PDT -- DEBUG:rotkehlchen.rotkehlchen:Main loop start
19/09/2018 21:24:53 PDT -- DEBUG:rotkehlchen.rotkehlchen:Maybe upload to server
19/09/2018 21:24:53 PDT -- DEBUG:rotkehlchen.rotkehlchen:Main loop end
19/09/2018 21:25:53 PDT -- DEBUG:rotkehlchen.rotkehlchen:Main loop start
19/09/2018 21:25:53 PDT -- DEBUG:rotkehlchen.rotkehlchen:Maybe upload to server
19/09/2018 21:25:53 PDT -- DEBUG:rotkehlchen.rotkehlchen:Main loop end
19/09/2018 21:26:53 PDT -- DEBUG:rotkehlchen.rotkehlchen:Main loop start
19/09/2018 21:26:53 PDT -- DEBUG:rotkehlchen.rotkehlchen:Maybe upload to server
19/09/2018 21:26:53 PDT -- DEBUG:rotkehlchen.rotkehlchen:Main loop end
19/09/2018 21:27:53 PDT -- DEBUG:rotkehlchen.rotkehlchen:Main loop start
19/09/2018 21:27:53 PDT -- DEBUG:rotkehlchen.rotkehlchen:Maybe upload to server
19/09/2018 21:27:53 PDT -- DEBUG:rotkehlchen.rotkehlchen:Main loop end
19/09/2018 21:28:53 PDT -- DEBUG:rotkehlchen.rotkehlchen:Main loop start
19/09/2018 21:28:53 PDT -- DEBUG:rotkehlchen.rotkehlchen:Maybe upload to server
19/09/2018 21:28:53 PDT -- DEBUG:rotkehlchen.rotkehlchen:Main loop end
19/09/2018 21:29:53 PDT -- DEBUG:rotkehlchen.rotkehlchen:Main loop start
19/09/2018 21:29:53 PDT -- DEBUG:rotkehlchen.rotkehlchen:Maybe upload to server
19/09/2018 21:29:53 PDT -- DEBUG:rotkehlchen.rotkehlchen:Main loop end

```


### System Description

```
KennysMBP:rotkehlchen ascherik$ uname -a
Darwin KennysMBP 17.7.0 Darwin Kernel Version 17.7.0: Thu Jun 21 22:53:14 PDT 2018; root:xnu-4570.71.2~1/RELEASE_X86_64 x86_64
KennysMBP:rotkehlchen ascherik$ command -v geth && geth version
/usr/local/bin/geth
Geth
Version: 1.8.15-stable
Architecture: amd64
Protocol Versions: [63 62]
Network Id: 1
Go Version: go1.11
Operating System: darwin
GOPATH=
GOROOT=/usr/local/opt/go/libexec
KennysMBP:rotkehlchen ascherik$ command -v parity && parity --version
KennysMBP:rotkehlchen ascherik$ command -v rotkehlchen && rotkehlchen version
KennysMBP:rotkehlchen ascherik$ [ -d .git ] && git rev-parse HEAD
8b9974c60373f9477581e234c2ab6366d45498e2
```


Discussion:
Hey @kascheri12 no they are not saved.

If you connect an ethereum node then it uses the ethereum node to query your balances. If the ethereum node is not synced then you will get the balances of the block height your syncing is at.

The proper solution here would be to revert back to etherscan querying if the ethereum node is not synced.

Resolving patch:
diff --git a/rotkehlchen/ethchain.py b/rotkehlchen/ethchain.py
--- a/rotkehlchen/ethchain.py
+++ b/rotkehlchen/ethchain.py
@@ -30,7 +30,7 @@ def attempt_connect(self, ethrpc_port, mainnet_check=True) -> Tuple[bool, str]:
         try:
             self.web3 = Web3(HTTPProvider('http://localhost:{}'.format(ethrpc_port)))
         except ConnectionError:
-            logger.warn('Could not connect to a local ethereum node. Will use etherscan only')
+            logger.warning('Could not connect to a local ethereum node. Will use etherscan only')
             self.connected = False
             return False, 'Failed to connect to ethereum node at port {}'.format(ethrpc_port)
 
@@ -44,7 +44,7 @@ def attempt_connect(self, ethrpc_port, mainnet_check=True) -> Tuple[bool, str]:
                 genesis_hash = self.web3.eth.getBlock(0)['hash'].hex()  # pylint: disable=no-member
                 target = '0xd4e56740f876aef8c010b86a40d5f56745a118d0906a34e69aec8c0db1cb8fa3'
                 if genesis_hash != target:
-                    logger.warn(
+                    logger.warning(
                         'Connected to a local ethereum node but it is not on the ethereum mainnet'
                     )
                     self.connected = False
@@ -53,17 +53,43 @@ def attempt_connect(self, ethrpc_port, mainnet_check=True) -> Tuple[bool, str]:
                         'the ethereum mainnet'.format(ethrpc_port)
                     )
                     return False, message
+                if self.web3.eth.syncing:  # pylint: disable=no-member
+                    current_block = self.web3.eth.syncing.currentBlock  # pylint: disable=no-member
+                    latest_block = self.web3.eth.syncing.highestBlock  # pylint: disable=no-member
+                    return self.is_synchronized(current_block, latest_block)
+                else:
+                    current_block = self.web3.eth.blockNumber  # pylint: disable=no-member
+                    latest_block = self.query_eth_highest_block()
+                    if latest_block is None:
+                        return False, 'Could not query latest block from blockcypher.'
+                    return self.is_synchronized(current_block, latest_block)
 
             self.connected = True
             return True, ''
         else:
-            logger.warn('Could not connect to a local ethereum node. Will use etherscan only')
+            logger.warning('Could not connect to a local ethereum node. Will use etherscan only')
             self.connected = False
             message = 'Failed to connect to ethereum node at port {}'.format(ethrpc_port)
 
         # If we get here we did not connnect
         return False, message
 
+    def is_synchronized(self, current_block: int, latest_block: int) -> Tuple[bool, str]:
+        """ Validate that the local ethereum node is synchronized
+            within 20 blocks of latest block
+
+        Returns a tuple (results, message)
+            - result: Boolean for confirmation of synchronized
+            - message: A message containing information on what the status is. """
+        if current_block < (latest_block - 20):
+            message = (
+                'Found local ethereum node but it is out of sync. Will use etherscan.'
+            )
+            logger.warning(message)
+            self.connected = False
+            return False, message
+        return True, message
+
     def set_rpc_port(self, port: int) -> Tuple[bool, str]:
         """ Attempts to set the RPC port for the ethereum client.
 
@@ -76,6 +102,19 @@ def set_rpc_port(self, port: int) -> Tuple[bool, str]:
             self.ethrpc_port = port
         return result, message
 
+    def query_eth_highest_block(self) -> int:
+        """ Attempts to query blockcypher for the block height
+
+        Returns the highest blockNumber"""
+
+        eth_resp = request_get(
+            'https://api.blockcypher.com/v1/eth/main'
+        )
+        if 'height' not in eth_resp:
+            return None
+        blockNumber = int(eth_resp['height'])
+        return blockNumber
+
     def get_eth_balance(self, account: typing.EthAddress) -> FVal:
         if not self.connected:
             eth_resp = request_get(


Test patch: