Repository: sunpy/sunpy

Issue:
Remote data manager does not recover well from its database being deleted while a session is running
<!-- This comments are hidden when you submit the issue so you do not need to remove them!
Please be sure to check out our contributing guidelines: https://github.com/sunpy/sunpy/blob/master/CONTRIBUTING.rst
Please be sure to check out our code of conduct:
https://github.com/sunpy/sunpy/blob/master/CODE_OF_CONDUCT.rst -->

<!-- Please have a search on our GitHub repository to see if a similar issue has already been posted.
If a similar issue is closed, have a quick look to see if you are satisfied by the resolution.
If not please go ahead and open an issue! -->

### Description
You can permanently corrupt the data manager state. So nothing works.

### Expected behavior
It recovers from my abuse and fixes itself.


### Steps to Reproduce
<!-- Ideally a code example could be provided so we can run it ourselves. -->
<!-- If you are pasting code, use tripe backticks (```) around your code snippet. -->

1. Delete all your data_manager stuff so you have a clean slate.
2. Run the following code inside an interactive prompt:
```python
from sunpy.data import manager
@manager.require('test_file',
                  ['http://data.sunpy.org/sample-data/predicted-sunspot-radio-flux.txt'],
                  '4c85b04a5528aa97eb84a087450eda0421c71833820576330bba148564089b11')
def test_function():
    pass

test_function()
```
3. Observe how everything works.
4. do a rm ~/sunpy/data_manager/*
5. **Without** starting a new python session run: `test_function()`, observe the following error:
```
---------------------------------------------------------------------------
OperationalError                          Traceback (most recent call last)
<ipython-input-2-7e96d146b416> in <module>
----> 1 test_function()

~/.virtualenvs/sunpy-release/lib/python3.8/site-packages/sunpy/data/data_manager/manager.py in wrapper(*args, **kwargs)
     64                     file_path = self._cache.download(urls, redownload=True)
     65                 else:
---> 66                     details = self._cache.get_by_hash(sha_hash)
     67                     if not details:
     68                         # In case we are matching by hash and file does not exist

~/.virtualenvs/sunpy-release/lib/python3.8/site-packages/sunpy/data/data_manager/cache.py in get_by_hash(self, sha_hash)
    127             SHA-1 hash of the file.
    128         """
--> 129         details = self._storage.find_by_key('file_hash', sha_hash)
    130         return details
    131 

~/.virtualenvs/sunpy-release/lib/python3.8/site-packages/sunpy/data/data_manager/storage.py in find_by_key(self, key, value)
    161         with self.connection() as conn:
    162             cursor = conn.cursor()
--> 163             cursor.execute(f'''SELECT * FROM {self._table_name}
    164                                       WHERE {key}="{value}"''')
    165             row = cursor.fetchone()

OperationalError: no such table: cache_storage

```
6. Restart your Python session and re-run:
```python
from sunpy.data import manager
@manager.require('test_file',
                  ['http://data.sunpy.org/sample-data/predicted-sunspot-radio-flux.txt'],
                  '4c85b04a5528aa97eb84a087450eda0421c71833820576330bba148564089b11')
def test_function():
    pass

test_function()
```
7. Observe how it's still broken.

Note, that if you delete the contents of the `data_manager` dir and then *restart* your Python session before calling the data manager again it recovers fine.

### System Details
<!-- We at least need to know the SunPy version you are using. -->
<!-- We provide a short function in SunPy that will provide some of the below information. -->
<!-- It is sunpy.util.system_info(), this is optional but strongly recommended. -->

  - SunPy Version: 1.1.1
  - Python Version: 3.8



Discussion:
I can work on this, could you give a me hint on how I should go about fixing this?
@amogh-jrules 

Two ways to fix this comes to my mind,

 1. https://github.com/sunpy/sunpy/blob/c6b475659c1c366c574e8b755e7a828c31b6182f/sunpy/data/data_manager/storage.py#L150

	moving this line to init, making it an attribute of the class `self._conn` and using `self._conn` instead of `conn` should fix this issue. 

2. You can also check for the exception at every `execute` call, remove the file and call setup again.

You can open a PR with either of these and we can discuss it over there. 
Thanks! I'll get started
@vn-ki I have a tiny doubt. So if I make conn an attribute of the SqliteStorage class and initialize in init, do I also connect with sqlite3 instance in init itself as in the code you mentioned?
Hi @Cadair, I worked on this a bit and seem to have fixed it. The tests pass locally. Creating a PR now.

Resolving patch:
diff --git a/sunpy/data/data_manager/storage.py b/sunpy/data/data_manager/storage.py
--- a/sunpy/data/data_manager/storage.py
+++ b/sunpy/data/data_manager/storage.py
@@ -122,16 +122,12 @@ def __init__(self, path):
             self._setup()
 
     def _setup(self):
-        schema = ' text, '.join(self.COLUMN_NAMES) + ' text'
         with self.connection(commit=True) as conn:
-            # Do this in a try...except to prevent race conditions in the tests
-            try:
-                conn.execute(f'''CREATE TABLE {self._table_name}
-                                ({schema})''')
-            except sqlite3.OperationalError as exc:
-                if "cache_storage already exists" in str(exc):
-                    return
-                raise exc
+            self._create_table(conn)
+
+    def _create_table(self, conn):
+        schema = ' text, '.join(self.COLUMN_NAMES) + ' text'
+        conn.execute(f'''CREATE TABLE IF NOT EXISTS {self._table_name} ({schema})''')
 
     @contextmanager
     def connection(self, commit=False):
@@ -144,6 +140,7 @@ def connection(self, commit=False):
             Whether to commit after succesful execution of db command.
         """
         conn = sqlite3.connect(str(self._db_path))
+        self._create_table(conn)
         try:
             yield conn
             if commit:


Test patch:
diff --git a/sunpy/data/data_manager/tests/conftest.py b/sunpy/data/data_manager/tests/conftest.py
--- a/sunpy/data/data_manager/tests/conftest.py
+++ b/sunpy/data/data_manager/tests/conftest.py
@@ -55,6 +55,16 @@ def manager(tmp_path, downloader, storage, mocker):
     yield manager
 
 
+@pytest.fixture
+def sqlmanager(tmp_path, downloader, sqlstorage, mocker):
+    sqlmanager = DataManager(Cache(downloader, sqlstorage, tmp_path))
+    sqlmanager._tempdir = str(tmp_path)
+    m = mock.Mock()
+    m.headers = {'Content-Disposition': 'test_file'}
+    mocker.patch('sunpy.data.data_manager.cache.urlopen', return_value=m)
+    yield sqlmanager
+
+
 @pytest.fixture
 def data_function(manager):
     @manager.require('test_file', ['url1/test_file', 'url2'], mocks.MOCK_HASH)
diff --git a/sunpy/data/data_manager/tests/test_manager.py b/sunpy/data/data_manager/tests/test_manager.py
--- a/sunpy/data/data_manager/tests/test_manager.py
+++ b/sunpy/data/data_manager/tests/test_manager.py
@@ -1,3 +1,4 @@
+import os
 from pathlib import Path
 
 import pytest
@@ -144,3 +145,18 @@ def test_file_changed(data_function, storage):
     # Now it should error
     with pytest.warns(SunpyUserWarning):
         data_function()
+
+
+def test_delete_db(sqlmanager, sqlstorage):
+    # Download the file
+    @sqlmanager.require('test_file', ['http://example.com/test_file'], MOCK_HASH)
+    def test_function():
+        pass
+
+    test_function()
+
+    # The DB file was then deleted
+    os.remove(str(sqlstorage._db_path))
+
+    # SQLite should not throw an error
+    test_function()