Repository: sunpy/sunpy

Issue:
Fixing flaky tests in `sunpy.database`
### ~Make errors from `Database.fetch` available?~
~I'm trying to investigate some flaky remote data tests. A lot of the tests rely on asserting the length of the database after fetching. Would it be useful (and make sense) if there was some way to access errors from the `Downloader` in a `Database` object? If we can access the error we should be able to print which servers are failing in the test log.  In the code below `paths._errors` should contain any errors.~

~https://github.com/sunpy/sunpy/blob/bf3a54f6efd104faad4f4bb7347ba542cb2f16a2/sunpy/database/database.py#L412-L414~

### Possible bug if only some downloads fail
Also, in the above code, could there ever be a situation where `paths` and `query_result` are not paired correctly by `zip`? If there is an error downloading a file, it will not appear in `paths`. I've only seen `paths=[]` which is fine, but could you get `len(paths)==1` and `len(query_result)==2`?

---
### Specific server that's not responding consistently
From multiple networks I'm finding that `sdo7.nascom.nasa.gov` is taking a long time to respond but eventually gives a 500 internal server error. The response provides an email address to report the error. Any point doing that? See example:

```sh
curl "https://sdo7.nascom.nasa.gov/cgi-bin/drms_export.cgi?series=aia__lev1;compress=rice;record=335_1123200039-1123200039"
```



Discussion:
1) Sounds good
2) I'm not sure?
3) Yeah, probably worth reporting the error if there's contact info.

Resolving patch:
diff --git a/sunpy/database/__init__.py b/sunpy/database/__init__.py
--- a/sunpy/database/__init__.py
+++ b/sunpy/database/__init__.py
@@ -10,6 +10,7 @@
     EntryAlreadyUnstarredError,
     EntryNotFoundError,
     NoSuchTagError,
+    PartialFetchError,
     TagAlreadyAssignedError,
     disable_undo,
     split_database,
@@ -19,4 +20,4 @@
     'Database', 'EntryAlreadyAddedError', 'NoSuchEntryError', 'NoSuchTagError',
     'NonRemovableTagError', 'EntryAlreadyStarredError',
     'EntryAlreadyUnstarredError', 'EntryNotFoundError',
-    'TagAlreadyAssignedError', 'disable_undo', 'split_database']
+    'TagAlreadyAssignedError', 'PartialFetchError', 'disable_undo', 'split_database']
diff --git a/sunpy/database/database.py b/sunpy/database/database.py
--- a/sunpy/database/database.py
+++ b/sunpy/database/database.py
@@ -121,6 +121,23 @@ def __str__(self):
         return errmsg.format(self.database_entry, self.tag_name)
 
 
+class PartialFetchError(Exception):
+    """
+    This exception is raised if the number of files returned by the
+    downloader does not match the number expected.
+    """
+
+    def __init__(self, paths, n_expected):
+        self.paths = paths
+        self.n_expected = int(n_expected)
+
+    def __str__(self):
+        return (f"The downloader returned {len(self.paths)} file(s) "
+                f"but {self.n_expected} file(s) were expected.\n"
+                f"Successful downloads: {self.paths}\n"
+                f"Errors: {self.paths.errors}")
+
+
 def split_database(source_database, destination_database, *query_string):
     """
     Queries the source database with the query string, and moves the
@@ -411,6 +428,10 @@ def _download_and_collect_entries(self, query_result, client=None,
 
         paths = client.fetch(query_result, path)
 
+        # Unable to handle partial success
+        if len(paths) != len(query_result):
+            raise PartialFetchError(paths, len(query_result))
+
         for (path, block) in zip(paths, query_result):
             qr_entry = tables.DatabaseEntry._from_query_result_block(block)
 


Test patch:
diff --git a/sunpy/database/tests/test_database.py b/sunpy/database/tests/test_database.py
--- a/sunpy/database/tests/test_database.py
+++ b/sunpy/database/tests/test_database.py
@@ -27,6 +27,7 @@
     EntryAlreadyUnstarredError,
     EntryNotFoundError,
     NoSuchTagError,
+    PartialFetchError,
     TagAlreadyAssignedError,
     attrs,
     disable_undo,
@@ -1038,6 +1039,22 @@ def test_fetch_separate_filenames():
     shutil.rmtree(tmp_test_dir)
 
 
+@pytest.mark.remote_data
+def test_fetch_partial_download(mocker, database, download_query, tmpdir):
+    results = Results()
+    results.append("successful_download.fits")
+    results.add_error("unsuccessful_download.fits", "https://timeout.invalid", "Timeout Error")
+    mocker.patch("sunpy.net.vso.VSOClient.fetch", return_value=results)
+    path = str(tmpdir.join('{file}.fits'))
+    with pytest.raises(PartialFetchError) as e:
+        database.fetch(*download_query, path=path)
+    successes, failures = str(e.value).split("Errors: ")
+    assert "successful_download.fits" in successes
+    assert "unsuccessful_download.fits" in failures
+    assert "https://timeout.invalid" in failures
+    assert "Timeout Error" in failures
+
+
 @pytest.mark.remote_data
 def test_disable_undo(database, download_query, tmpdir):
     entry = DatabaseEntry()