Repository: astropy/astropy

Issue:
IERSRangeError after downloading table from USNO
<!-- This comments are hidden when you submit the issue,
so you do not need to remove them! -->

<!-- Please be sure to check out our contributing guidelines,
https://github.com/astropy/astropy/blob/main/CONTRIBUTING.md .
Please be sure to check out our code of conduct,
https://github.com/astropy/astropy/blob/main/CODE_OF_CONDUCT.md . -->

<!-- Please have a search on our GitHub repository to see if a similar
issue has already been posted.
If a similar issue is closed, have a quick look to see if you are satisfied
by the resolution.
If not please go ahead and open an issue! -->

<!-- Please check that the development version still produces the same bug.
You can install development version with
pip install git+https://github.com/astropy/astropy
command. -->

### Description
Recent dates don't seem to be included in the IERS table when I force a fresh download

### Expected behavior
I expected to convert a recent date

### Actual behavior
astropy.utils.iers.iers.IERSRangeError: (some) times are outside of range covered by IERS table.

### Steps to Reproduce
<!-- Ideally a code example could be provided so we can run it ourselves. -->
<!-- If you are pasting code, use triple backticks (```) around
your code snippet. -->
<!-- If necessary, sanitize your screen output to be pasted so you do not
reveal secrets like tokens and passwords. -->

```python
# Put your Python code snippet here.
from astropy.utils.data import download_file
from astropy.time import Time, TimeDelta
from astropy.utils import iers
url = 'https://maia.usno.navy.mil/ser7/finals2000A.all'
filename = download_file(url, cache=False)  # Force download of latest copy
dat = iers.IERS_Auto.read(file=filename)
iers.IERS.iers_table = dat
iers.conf.auto_download = False
(Time.now()-TimeDelta(3)).ut1 #Raises Error
# astropy.utils.iers.iers.IERSRangeError: (some) times are outside of range covered by IERS table.
for i in range(1000):
    try:
        LastDate = (Time('2021-01-11')+TimeDelta(i)).ut1
    except iers.iers.IERSRangeError:
        print(Time('2021-01-11')+TimeDelta(i))
        break
# 2022-02-06 00:00:00.000 is the first date that cannot be converted


```

### System Details
<!-- Even if you do not think this is necessary, it is useful information for the maintainers.
Please run the following snippet and paste the output below:
import platform; print(platform.platform())
import sys; print("Python", sys.version)
import numpy; print("Numpy", numpy.__version__)
import erfa; print("pyerfa", erfa.__version__)
import astropy; print("astropy", astropy.__version__)
import scipy; print("Scipy", scipy.__version__)
import matplotlib; print("Matplotlib", matplotlib.__version__)
-->
Python 3.9.12 (main, Apr  5 2022, 06:56:58)
[GCC 7.5.0]
Numpy 1.21.5
pyerfa 2.0.0
astropy 5.0.4
Scipy 1.7.3
Matplotlib 3.5.1


Discussion:
github-actions[bot]: 👋 Thank you for your draft pull request! Do you know that you can use `[ci skip]` or `[skip ci]` in your commit messages to skip running continuous integration tests until you are ready?
neutrinoceros: Just a heads-up that this patch seems to touch some of the same lines than #16070, so merge conflicts might ensue. That said, my own PR is currently stuck so I don't have a problem with this one going in first, mind you. I'm also happy to serve as a reviewer if needed !
ayshih: I think #16070 will be sufficiently orthogonal – improving how IERS-A and IERS-B are combined versus merely ensuring that the combination even happens – that any merge conflicts should be trivial to resolve.
mhvk: @ayshih - yes, good point. So, I guess to get this to work one would have to also change `_refresh_table_as_needed` and in the download failure path check whether `IERS_A_FILE` is more up to date then the current one: effectively, for the no download case, one could do `new_table = cls.read()` there and fall through to the part that updates `self` as needed.

p.s. We can obviously move this to a new issue/PR!
ayshih: Let's get this PR in to solve the easy-to-encounter bug, and defer the improved handling of two IERS-A files to a future PR.  I've updated the documentation more, including https://docs.astropy.org/en/stable/utils/data.html

Patch:
diff --git a/astropy/utils/iers/iers.py b/astropy/utils/iers/iers.py
index 6e14c786fb..04ff0efdb4 100644
--- a/astropy/utils/iers/iers.py
+++ b/astropy/utils/iers/iers.py
@@ -165,9 +165,9 @@ class Conf(_config.ConfigNamespace):
 
     auto_download = _config.ConfigItem(
         True,
-        "Enable auto-downloading of the latest IERS data.  If set to False "
-        "then the local IERS-B file will be used by default (even if the "
-        "full IERS file with predictions was already downloaded and cached). "
+        "Enable auto-downloading of the latest IERS-A data.  If set to False "
+        "then the bundled IERS-A file will be used by default (even if a "
+        "newer version of the IERS-A file was previously downloaded and cached). "
         "This parameter also controls whether internet resources will be "
         "queried to update the leap second table if the installed version is "
         "out of date. Default is True.",
@@ -191,7 +191,7 @@ class Conf(_config.ConfigNamespace):
         ["error", "warn", "ignore"],
         "IERS behavior if the range of available IERS data does not "
         "cover the times when converting time scales, potentially leading "
-        "to degraded accuracy.",
+        "to degraded accuracy.  Applies only when using IERS-B data on its own.",
     )
     system_leap_second_file = _config.ConfigItem("", "System file with leap seconds.")
     iers_leap_second_auto_url = _config.ConfigItem(
@@ -545,9 +545,9 @@ class IERS_A(IERS):
 
     Notes
     -----
-    The IERS A file is not part of astropy.  It can be downloaded from
-    ``iers.IERS_A_URL`` or ``iers.IERS_A_URL_MIRROR``. See ``iers.__doc__``
-    for instructions on use in ``Time``, etc.
+    If the package IERS A file (``iers.IERS_A_FILE``) is out of date, a new
+    version can be downloaded from ``iers.IERS_A_URL`` or ``iers.IERS_A_URL_MIRROR``.
+    See ``iers.__doc__`` for instructions on use in ``Time``, etc.
     """
 
     iers_table = None
@@ -684,7 +684,7 @@ class IERS_B(IERS):
 
     Notes
     -----
-    If the package IERS B file (```iers.IERS_B_FILE``) is out of date, a new
+    If the package IERS B file (``iers.IERS_B_FILE``) is out of date, a new
     version can be downloaded from ``iers.IERS_B_URL``.
 
     See `~astropy.utils.iers.IERS_B.read` for instructions on how to read
@@ -758,6 +758,10 @@ class IERS_Auto(IERS_A):
     """
     Provide most-recent IERS data and automatically handle downloading
     of updated values as necessary.
+
+    The returned table combines the IERS-A and IERS-B files, with the data
+    in the IERS-B file considered to be official values and thus superseding
+    values from the IERS-A file at the same times.
     """
 
     iers_table = None
@@ -772,8 +776,8 @@ class IERS_Auto(IERS_A):
         (or non-existent) then it will be downloaded over the network and cached.
 
         If the configuration setting ``astropy.utils.iers.conf.auto_download``
-        is set to False then ``astropy.utils.iers.IERS()`` is returned.  This
-        is normally the IERS-B table that is supplied with astropy.
+        is set to False then the bundled IERS-A table will be used rather than
+        any downloaded version of the IERS-A table.
 
         On the first call in a session, the table will be memoized (in the
         ``iers_table`` class attribute), and further calls to ``open`` will
@@ -786,7 +790,7 @@ class IERS_Auto(IERS_A):
 
         """
         if not conf.auto_download:
-            cls.iers_table = IERS_B.open()
+            cls.iers_table = cls.read()
             return cls.iers_table
 
         all_urls = (conf.iers_auto_url, conf.iers_auto_url_mirror)
@@ -817,8 +821,10 @@ class IERS_Auto(IERS_A):
             # Issue a warning here, perhaps user is offline.  An exception
             # will be raised downstream if actually trying to interpolate
             # predictive values.
-            warn("unable to download valid IERS file, using local IERS-B", IERSWarning)
-            cls.iers_table = IERS_B.open()
+            warn(
+                "unable to download valid IERS file, using bundled IERS-A", IERSWarning
+            )
+            cls.iers_table = cls.read()
 
         return cls.iers_table
 
@@ -932,7 +938,7 @@ class IERS_Auto(IERS_A):
         IERS-A has IERS-B values included, but for reasons unknown these
         do not match the latest IERS-B values (see comments in #4436).
         Here, we use the bundled astropy IERS-B table to overwrite the values
-        in the downloaded IERS-A table.
+        in the IERS-A table.
         """
         iers_b = IERS_B.open()
         # Substitute IERS-B values for existing B values in IERS-A table
diff --git a/docs/changes/utils/16187.api.rst b/docs/changes/utils/16187.api.rst
new file mode 100644
index 0000000000..f5eccb2bf7
--- /dev/null
+++ b/docs/changes/utils/16187.api.rst
@@ -0,0 +1,5 @@
+``IERS_Auto.open()`` now always returns a table of type ``IERS_Auto`` that
+contains the combination of IERS-A and IERS-B data, even if automatic
+updating of the IERS-A file is disabled or if downloading the new file fails.
+Previously, under those conditions, it would return a table of a different type
+(``IERS_B``) with only IERS-B data.
diff --git a/docs/changes/utils/16187.bugfix.rst b/docs/changes/utils/16187.bugfix.rst
new file mode 100644
index 0000000000..50d7d578f2
--- /dev/null
+++ b/docs/changes/utils/16187.bugfix.rst
@@ -0,0 +1,1 @@
+Fixed the unintended behavior where the IERS-A file bundled in ``astropy-iers-data`` would be ignored if automatic updating of the IERS-A file were disabled or if downloading the new file failed.
diff --git a/docs/utils/data.rst b/docs/utils/data.rst
index a9bbf1e440..b99db19234 100644
--- a/docs/utils/data.rst
+++ b/docs/utils/data.rst
@@ -9,8 +9,7 @@ Introduction
 
 A number of Astropy's tools work with data sets that are either awkwardly
 large (e.g., `~astropy.coordinates.solar_system_ephemeris`) or
-regularly updated (e.g., `~astropy.utils.iers.IERS_B`) or both
-(e.g., `~astropy.utils.iers.IERS_A`). This kind of
+regularly updated (e.g., `~astropy.utils.iers.IERS_A`). This kind of
 data - authoritative data made available on the Web, and possibly updated
 from time to time - is reasonably common in astronomy. The Astropy Project therefore
 provides some tools for working with such data.
@@ -235,9 +234,8 @@ adding files to an existing cache directory.
 If your application needs IERS data specifically, you can download the
 appropriate IERS table, covering the appropriate time span, by any means you
 find convenient. You can then load this file into your application and use the
-resulting table rather than `~astropy.utils.iers.IERS_Auto`. In fact, the IERS
-B table is small enough that a version (not necessarily recent) is bundled with
-``astropy`` as ``astropy.utils.iers.IERS_B_FILE``. Using a specific non-automatic
+resulting table rather than `~astropy.utils.iers.IERS_Auto`.
+Using a specific non-automatic
 table also has the advantage of giving you control over exactly which version
 of the IERS data your application is using. See also :ref:`iers-working-offline`.
 
@@ -265,25 +263,19 @@ systems. The parallel access to the home directory can also trigger concurrency
 problems in the Astropy data cache, though we have tried to minimize these. We
 therefore recommend the following guidelines:
 
- * Do one of the following:
+ * Set ``astropy.utils.iers.conf.auto_download = False`` in your ``astropy`` config
+   file (see :ref:`astropy_config`) or in your code so that ``astropy`` will not
+   attempt to download a newer version of the IERS-A table than the one already
+   bundled in the ``astropy-iers-data`` package.  To update the IERS-A table, do
+   one of the following:
 
+   * Upgrade the ``astropy-iers-data`` package.
    * Write a simple script that sets ``astropy.utils.iers.conf.auto_download =
      True`` and then accesses all cached resources your code will need,
      including source name lookups and IERS tables. Run it on the head node from
      time to time (frequently enough to beat the timeout
      ``astropy.utils.iers.conf.auto_max_age``, which defaults to 30 days) to
      ensure all data is up to date.
-   * Set ``astropy.utils.iers.conf.auto_download = False`` in your code and set
-     ``astropy.utils.iers.conf.iers_degraded_accuracy`` to either ``'warn'``
-     or ``'ignore'``. These prevent the normal exception that occurs if a
-     time conversion falls outside the bounds of available (local) IERS data.
-     **WARNING**: only use this option if your application does not need full
-     accuracy time conversions.
-
- * Make an Astropy config file (see :ref:`astropy_config`) that sets
-   ``astropy.utils.iers.conf.auto_download = False`` so that the worker jobs will
-   not suddenly notice an out-of-date table all at once and frantically attempt
-   to download it.
 
  * Optionally, in this file, set ``astropy.utils.data.conf.allow_internet = False`` to
    prevent any attempt to download any file from the worker nodes; if you do this,
diff --git a/docs/utils/iers.rst b/docs/utils/iers.rst
index 411f003bce..3ee9d7b2d5 100644
--- a/docs/utils/iers.rst
+++ b/docs/utils/iers.rst
@@ -94,9 +94,9 @@ that relate to automatic IERS downloading. Four of the most
 important to consider are the following:
 
   auto_download:
-    Enable auto-downloading of the latest IERS data.  If set to ``False`` then
-    the local IERS-A and IERS-B files will be used by default (even if the full
-    IERS file with predictions was already downloaded and cached).  This
+    Enable auto-downloading of the latest IERS-A data.  If set to ``False`` then
+    the bundled IERS-A file will be used by default (even if a newer
+    versions of the IERS-A file was already downloaded and cached).  This
     parameter also controls whether internet resources will be queried to update
     the leap second table if the installed version is out of date.
 
@@ -109,12 +109,10 @@ important to consider are the following:
 
   iers_degraded_accuracy:
     Some time conversions like UTC -> UT1 require IERS-A Earth rotation data
-    for full accuracy. In cases where full accuracy is not required and
-    downloading the IERS-A is not possible or desired (for instance running on
-    a cluster) then this option can be set to either ``'warn'`` or ``'ignore'``.
-    The default is ``'error'`` which will raise an exception if full accuracy
-    is not possible for a time conversion, ``'warn'`` will issue a warning, and
-    ``'ignore'`` will ignore the problem and use available IERS-B data.
+    for full accuracy.  This parameter controls the behavior when computations
+    use only the IERS-B data and full accuracy is not possible.  ``'error'``
+    (the default) will raise an exception, ``'warn'`` will issue a warning, and
+    ``'ignore'`` will ignore the problem (i.e., the inaccuracy is acceptable).
 
 Auto refresh behavior
 ---------------------


Test patch:
diff --git a/astropy/coordinates/tests/test_iau_fullstack.py b/astropy/coordinates/tests/test_iau_fullstack.py
index 32635bfa7f..bb162e5ac3 100644
--- a/astropy/coordinates/tests/test_iau_fullstack.py
+++ b/astropy/coordinates/tests/test_iau_fullstack.py
@@ -197,28 +197,16 @@ def test_future_altaz():
     location = EarthLocation(lat=0 * u.deg, lon=0 * u.deg)
     t = Time("J2161")
 
-    # check that these message(s) appear among any other warnings.  If tests are run with
-    # --remote-data then the IERS table will be an instance of IERS_Auto which is
-    # assured of being "fresh".  In this case getting times outside the range of the
-    # table does not raise an exception.  Only if using IERS_B (which happens without
-    # --remote-data, i.e. for all CI testing) do we expect another warning.
+    # check that these message(s) appear among any other warnings
     if PYTEST_LT_8_0:
-        ctx1 = ctx2 = nullcontext()
+        ctx = nullcontext()
     else:
-        ctx1 = pytest.warns(erfa.core.ErfaWarning)
-        ctx2 = pytest.warns(AstropyWarning, match=".*times are outside of range.*")
+        ctx = pytest.warns(erfa.core.ErfaWarning)
     with (
-        ctx1,
-        ctx2,
+        ctx,
         pytest.warns(
             AstropyWarning,
             match="Tried to get polar motions for times after IERS data is valid.*",
-        ) as found_warnings,
+        ),
     ):
         SkyCoord(1 * u.deg, 2 * u.deg).transform_to(AltAz(location=location, obstime=t))
-
-    if isinstance(iers.earth_orientation_table.get(), iers.IERS_B):
-        assert any(
-            "(some) times are outside of range covered by IERS table." in str(w.message)
-            for w in found_warnings
-        )
diff --git a/astropy/utils/iers/tests/test_iers.py b/astropy/utils/iers/tests/test_iers.py
index 73bb3dda2b..f1cf188446 100644
--- a/astropy/utils/iers/tests/test_iers.py
+++ b/astropy/utils/iers/tests/test_iers.py
@@ -25,27 +25,9 @@ from astropy.utils.iers import iers
 
 FILE_NOT_FOUND_ERROR = getattr(__builtins__, "FileNotFoundError", OSError)
 
-try:
-    iers.IERS_A.open("finals2000A.all")  # check if IERS_A is available
-except OSError:
-    HAS_IERS_A = False
-else:
-    HAS_IERS_A = True
-
 IERS_A_EXCERPT = get_pkg_data_filename(os.path.join("data", "iers_a_excerpt"))
 
 
-def setup_module():
-    # Need auto_download so that IERS_B won't be loaded and cause tests to
-    # fail. Files to be downloaded are handled appropriately in the tests.
-    iers.conf.auto_download = True
-
-
-def teardown_module():
-    # This setting is to be consistent with astropy/conftest.py
-    iers.conf.auto_download = False
-
-
 class TestBasic:
     """Basic tests that IERS_B returns correct values"""
 
@@ -216,7 +198,6 @@ class TestIERS_AExcerpt:
         assert len(iers_tab[:2]) == 2
 
 
-@pytest.mark.skipif(not HAS_IERS_A, reason="requires IERS_A")
 class TestIERS_A:
     @classmethod
     def teardown_class(self):
@@ -261,6 +242,16 @@ class TestIERS_Auto:
         self.iers_a_url_2 = Path(self.iers_a_file_2).as_uri()
         self.t = Time.now() + TimeDelta(10, format="jd") * np.arange(self.N)
 
+        # This group of tests requires auto downloading to be on
+        self._auto_download = iers.conf.auto_download
+        iers.conf.auto_download = True
+
+        # auto_download = False is tested in test_IERS_B_parameters_loading_into_IERS_Auto()
+
+    def teardown_class(self):
+        # Restore the auto downloading setting
+        iers.conf.auto_download = self._auto_download
+
     def teardown_method(self, method):
         """Run this after every test."""
         iers.IERS_Auto.close()
@@ -310,12 +301,6 @@ class TestIERS_Auto:
                     iers_table = iers.IERS_Auto.open()
                     _ = iers_table.ut1_utc(self.t.jd1, self.t.jd2)
 
-    def test_no_auto_download(self):
-        with iers.conf.set_temp("auto_download", False):
-            t = iers.IERS_Auto.open()
-        assert type(t) is iers.IERS_B
-
-    @pytest.mark.remote_data
     def test_simple(self):
         with iers.conf.set_temp("iers_auto_url", self.iers_a_url_1):
             dat = iers.IERS_Auto.open()
@@ -383,9 +368,10 @@ class TestIERS_Auto:
             assert dat["MJD"][-1] == (57539.0 + 60) * u.d
 
 
-@pytest.mark.remote_data
 def test_IERS_B_parameters_loading_into_IERS_Auto():
-    A = iers.IERS_Auto.open()
+    # Make sure that auto downloading is off
+    with iers.conf.set_temp("auto_download", False):
+        A = iers.IERS_Auto.open()
     B = iers.IERS_B.open()
 
     ok_A = A["MJD"] <= B["MJD"][-1]
@@ -448,35 +434,31 @@ def test_iers_b_dl():
         iers.IERS_B.close()
 
 
-@pytest.mark.remote_data
-def test_iers_out_of_range_handling(tmp_path):
-    # Make sure we don't have IERS-A data available anywhere
-    with set_temp_cache(tmp_path):
-        iers.IERS_A.close()
-        iers.IERS_Auto.close()
-        iers.IERS.close()
+def test_iers_b_out_of_range_handling():
+    # The following error/warning applies only to IERS_B, not to the default IERS_Auto
+    with iers.earth_orientation_table.set(iers.IERS_B.open()):
         now = Time.now()
-        with iers.conf.set_temp("auto_download", False):
-            # Should be fine with built-in IERS_B
-            (now - 300 * u.day).ut1
 
-            # Default is to raise an error
-            match = r"\(some\) times are outside of range covered by IERS table"
-            with pytest.raises(iers.IERSRangeError, match=match):
-                (now + 100 * u.day).ut1
+        # Should be fine with bundled IERS-B
+        (now - 300 * u.day).ut1
 
-            with iers.conf.set_temp("iers_degraded_accuracy", "warn"):
-                with pytest.warns(iers.IERSDegradedAccuracyWarning, match=match):
-                    (now + 100 * u.day).ut1
+        # Default is to raise an error
+        match = r"\(some\) times are outside of range covered by IERS table"
+        with pytest.raises(iers.IERSRangeError, match=match):
+            (now + 100 * u.day).ut1
 
-            with iers.conf.set_temp("iers_degraded_accuracy", "ignore"):
+        with iers.conf.set_temp("iers_degraded_accuracy", "warn"):
+            with pytest.warns(iers.IERSDegradedAccuracyWarning, match=match):
                 (now + 100 * u.day).ut1
 
+        with iers.conf.set_temp("iers_degraded_accuracy", "ignore"):
+            (now + 100 * u.day).ut1
+
 
 @pytest.mark.remote_data
 def test_iers_download_error_handling(tmp_path):
-    # Make sure we don't have IERS-A data available anywhere
-    with set_temp_cache(tmp_path):
+    # Make sure an IERS-A table isn't already loaded
+    with set_temp_cache(tmp_path), iers.conf.set_temp("auto_download", True):
         iers.IERS_A.close()
         iers.IERS_Auto.close()
         iers.IERS.close()
@@ -488,7 +470,7 @@ def test_iers_download_error_handling(tmp_path):
             with iers.conf.set_temp("iers_auto_url_mirror", "https://google.com"):
                 with pytest.warns(iers.IERSWarning) as record:
                     with iers.conf.set_temp("iers_degraded_accuracy", "ignore"):
-                        (now + 100 * u.day).ut1
+                        (now + 400 * u.day).ut1
 
                 assert len(record) == 3
                 assert str(record[0].message).startswith(
@@ -498,7 +480,7 @@ def test_iers_download_error_handling(tmp_path):
                     "malformed IERS table from https://google.com"
                 )
                 assert str(record[2].message).startswith(
-                    "unable to download valid IERS file, using local IERS-B"
+                    "unable to download valid IERS file, using local IERS-A"
                 )