Repository: rotki/rotki

Issue:
New rotki accounts made from vXX and on may not have a DB version set if opened only once
## Problem Definition

After some changes we stopped writing the DB version in the DB for fresh accounts in their first run. The second run will write the current version to the DB.

That's since we bail here:
https://github.com/rotki/rotki/blob/f58859e7ff9f59a59c2b12142b48754861da4867/rotkehlchen/db/upgrade_manager.py#L230

and not get to the end where the version is written here:
https://github.com/rotki/rotki/blob/f58859e7ff9f59a59c2b12142b48754861da4867/rotkehlchen/db/upgrade_manager.py#L240-L245

Not having the actual current DB version written in the DB is quite problematic (even if it's just for the first run only) since this messes with DB upgrades.

Scenario:

1. User opens rotki v1.22.1 which has this bug, and does some stuff and closes it. Current DB is not written in the code.
2. If user downloads v1.23.0 which will contain DB upgrade v30->v31, during the DB initialization, DB upgrade will be totally skipped since rotki assumes latest DB if there is no setting set.
3. :boom: 

If in between 1 and 2 user opens rotki again we are safe. So this is kind of an edge case, but still possible to happen and a nasty bug which I think I introduced at some point.

### Task

- Find out at what point this bug was introduced, to know all affected DB versions. Figure out `XX`.
- Fix it. It's very easy.
- In develop, and specifically if there is no DB version set have some extra, but temporary code to sanity check the user DB and make sure that it is indeed the version we think. This can happen by checking if tables that exist after an upgrade, are there.
New rotki accounts made from vXX and on may not have a DB version set if opened only once
## Problem Definition

After some changes we stopped writing the DB version in the DB for fresh accounts in their first run. The second run will write the current version to the DB.

That's since we bail here:
https://github.com/rotki/rotki/blob/f58859e7ff9f59a59c2b12142b48754861da4867/rotkehlchen/db/upgrade_manager.py#L230

and not get to the end where the version is written here:
https://github.com/rotki/rotki/blob/f58859e7ff9f59a59c2b12142b48754861da4867/rotkehlchen/db/upgrade_manager.py#L240-L245

Not having the actual current DB version written in the DB is quite problematic (even if it's just for the first run only) since this messes with DB upgrades.

Scenario:

1. User opens rotki v1.22.1 which has this bug, and does some stuff and closes it. Current DB is not written in the code.
2. If user downloads v1.23.0 which will contain DB upgrade v30->v31, during the DB initialization, DB upgrade will be totally skipped since rotki assumes latest DB if there is no setting set.
3. :boom: 

If in between 1 and 2 user opens rotki again we are safe. So this is kind of an edge case, but still possible to happen and a nasty bug which I think I introduced at some point.

### Task

- Find out at what point this bug was introduced, to know all affected DB versions. Figure out `XX`.
- Fix it. It's very easy.
- In develop, and specifically if there is no DB version set have some extra, but temporary code to sanity check the user DB and make sure that it is indeed the version we think. This can happen by checking if tables that exist after an upgrade, are there.


Discussion:
So the commit this was introduced seem to be this: https://github.com/rotki/rotki/commit/f9bcf78d56c16e292a787dee9fc2398a5e53a521 by your truly. Fixed one big problem, introduce a smaller bug :sweat: 

The DB version at the time that code was written is v29: https://github.com/rotki/rotki/blob/f9bcf78d56c16e292a787dee9fc2398a5e53a521/rotkehlchen/db/settings.py#L18

So version upgrades that may have been affected would be v29 -> v30 and when v30->v31 is made in rotki v1.23.0 that too.
So the commit this was introduced seem to be this: https://github.com/rotki/rotki/commit/f9bcf78d56c16e292a787dee9fc2398a5e53a521 by your truly. Fixed one big problem, introduce a smaller bug :sweat: 

The DB version at the time that code was written is v29: https://github.com/rotki/rotki/blob/f9bcf78d56c16e292a787dee9fc2398a5e53a521/rotkehlchen/db/settings.py#L18

So version upgrades that may have been affected would be v29 -> v30 and when v30->v31 is made in rotki v1.23.0 that too.

Resolving patch:
diff --git a/rotkehlchen/db/dbhandler.py b/rotkehlchen/db/dbhandler.py
--- a/rotkehlchen/db/dbhandler.py
+++ b/rotkehlchen/db/dbhandler.py
@@ -318,9 +318,15 @@ def _run_actions_after_first_connection(self, password: str) -> None:
                 ) from e
 
         # Run upgrades if needed
-        DBUpgradeManager(self).run_upgrades()
+        fresh_db = DBUpgradeManager(self).run_upgrades()
         # create tables if needed (first run - or some new tables)
         self.conn.executescript(DB_SCRIPT_CREATE_TABLES)
+        if fresh_db:  # add DB version. https://github.com/rotki/rotki/issues/3744
+            cursor = self.conn.cursor()
+            cursor.execute(
+                'INSERT OR REPLACE INTO settings(name, value) VALUES(?, ?)',
+                ('version', str(ROTKEHLCHEN_DB_VERSION)),
+            )
 
     def get_md5hash(self) -> str:
         """Get the md5hash of the DB
diff --git a/rotkehlchen/db/upgrade_manager.py b/rotkehlchen/db/upgrade_manager.py
--- a/rotkehlchen/db/upgrade_manager.py
+++ b/rotkehlchen/db/upgrade_manager.py
@@ -214,9 +214,11 @@ class DBUpgradeManager():
     def __init__(self, db: 'DBHandler'):
         self.db = db
 
-    def run_upgrades(self) -> None:
+    def run_upgrades(self) -> bool:
         """Run all required database upgrades
 
+        Returns true for fresh database and false otherwise.
+
         May raise:
         - DBUpgradeError if the user uses a newer version than the one we
         upgrade to or if there is a problem during upgrade.
@@ -224,7 +226,7 @@ def run_upgrades(self) -> None:
         try:
             our_version = self.db.get_version()
         except sqlcipher.OperationalError:  # pylint: disable=no-member
-            return  # fresh database. Nothing to upgrade.
+            return True  # fresh database. Nothing to upgrade.
 
         if our_version > ROTKEHLCHEN_DB_VERSION:
             raise DBUpgradeError(
@@ -243,6 +245,7 @@ def run_upgrades(self) -> None:
             ('version', str(ROTKEHLCHEN_DB_VERSION)),
         )
         self.db.conn.commit()
+        return False
 
     def _perform_single_upgrade(self, upgrade: UpgradeRecord) -> None:
         """


Test patch:
diff --git a/rotkehlchen/tests/db/test_db.py b/rotkehlchen/tests/db/test_db.py
--- a/rotkehlchen/tests/db/test_db.py
+++ b/rotkehlchen/tests/db/test_db.py
@@ -1328,3 +1328,18 @@ def test_binance_pairs(user_data_dir):
     db.set_binance_pairs('binance', [], Location.BINANCE)
     query = db.get_binance_pairs('binance', Location.BINANCE)
     assert query == []
+
+
+def test_fresh_db_adds_version(user_data_dir):
+    """Test that the DB version gets committed to a fresh DB.
+
+    Regression test for https://github.com/rotki/rotki/issues/3744"""
+    msg_aggregator = MessagesAggregator()
+    db = DBHandler(user_data_dir, '123', msg_aggregator, None)
+    cursor = db.conn.cursor()
+    query = cursor.execute(
+        'SELECT value FROM settings WHERE name=?;', ('version',),
+    )
+    query = query.fetchall()
+    assert len(query) != 0
+    assert int(query[0][0]) == ROTKEHLCHEN_DB_VERSION