Repository: eWaterCycle/era5cli

Issue:
No error message if using --land and --ensemble 
While I was changing the test for `--land` and `--prelimbe` flags, I have realized that a request for a combination of `--land` and `--ensemble` goes through and [https://github.com/eWaterCycle/era5cli/blob/main/era5cli/fetch.py](url) doesn't raise an error. Should I add it, since I'm anyway working on it? 

`era5cli monthly --startyear 1990 --endyear 1990 --variables total_precipitation --land --ensemble`

Download request is being queued at Copernicus.
It can take some time before downloading starts, please do not kill this process in the meantime.

2022-10-24 16:14:54,411 INFO Welcome to the CDS
2022-10-24 16:14:54,411 INFO Sending request to https://cds.climate.copernicus.eu/api/v2/resources/reanalysis-era5-land-monthly-means
2022-10-24 16:14:54,784 INFO Request is queued
2022-10-24 16:14:55,954 INFO Request is running
2022-10-24 16:14:57,625 INFO Request is failed
2022-10-24 16:14:57,625 ERROR Message: no data is available within your requested subset
2022-10-24 16:14:57,625 ERROR Reason:  Request returned no data
2022-10-24 16:14:57,626 ERROR   Traceback (most recent call last):
2022-10-24 16:14:57,626 ERROR     File "/opt/cdstoolbox/cdscompute/cdscompute/cdshandlers/services/handler.py", line 59, in handle_request
2022-10-24 16:14:57,626 ERROR       result = cached(context.method, proc, context, context.args, context.kwargs)
2022-10-24 16:14:57,626 ERROR     File "/opt/cdstoolbox/cdscompute/cdscompute/caching.py", line 108, in cached
2022-10-24 16:14:57,626 ERROR       result = proc(context, *context.args, **context.kwargs)
2022-10-24 16:14:57,626 ERROR     File "/opt/cdstoolbox/cdscompute/cdscompute/services.py", line 124, in __call__
2022-10-24 16:14:57,626 ERROR       return p(*args, **kwargs)
2022-10-24 16:14:57,626 ERROR     File "/opt/cdstoolbox/cdscompute/cdscompute/services.py", line 60, in __call__
2022-10-24 16:14:57,626 ERROR       return self.proc(context, *args, **kwargs)
2022-10-24 16:14:57,626 ERROR     File "/home/cds/cdsservices/services/mars/mars.py", line 47, in internal
2022-10-24 16:14:57,626 ERROR       return mars(context, request, **kwargs)
2022-10-24 16:14:57,626 ERROR     File "/home/cds/cdsservices/services/mars/mars.py", line 19, in mars
2022-10-24 16:14:57,626 ERROR       execute_mars(context, requests)
2022-10-24 16:14:57,626 ERROR     File "/home/cds/cdsservices/services/mars/execute_mars.py", line 25, in execute_mars
2022-10-24 16:14:57,626 ERROR       raise NoDataException("Request returned no data", '')
2022-10-24 16:14:57,626 ERROR   cdsinf.exceptions.NoDataException: Request returned no data
Traceback (most recent call last):
  File "/home/acrnemr/miniconda3/envs/era5cl/bin/era5cli", line 8, in <module>
    sys.exit(main())
  File "/home/acrnemr/miniconda3/envs/era5cl/lib/python3.10/site-packages/era5cli/cli.py", line 435, in main
    _execute(args)
  File "/home/acrnemr/miniconda3/envs/era5cl/lib/python3.10/site-packages/era5cli/cli.py", line 425, in _execute
    era5.fetch(dryrun=args.dryrun)
  File "/home/acrnemr/miniconda3/envs/era5cl/lib/python3.10/site-packages/era5cli/fetch.py", line 176, in fetch
    self._split_variable_yr()
  File "/home/acrnemr/miniconda3/envs/era5cl/lib/python3.10/site-packages/era5cli/fetch.py", line 249, in _split_variable_yr
    pool.map(self._getdata, variables, years, outputfiles)
  File "/home/acrnemr/miniconda3/envs/era5cl/lib/python3.10/site-packages/pathos/threading.py", line 136, in map
    return _pool.map(star(f), zip(*args)) # chunksize
  File "/home/acrnemr/miniconda3/envs/era5cl/lib/python3.10/site-packages/multiprocess/pool.py", line 364, in map
    return self._map_async(func, iterable, mapstar, chunksize).get()
  File "/home/acrnemr/miniconda3/envs/era5cl/lib/python3.10/site-packages/multiprocess/pool.py", line 771, in get
    raise self._value
  File "/home/acrnemr/miniconda3/envs/era5cl/lib/python3.10/site-packages/multiprocess/pool.py", line 125, in worker
    result = (True, func(*args, **kwds))
  File "/home/acrnemr/miniconda3/envs/era5cl/lib/python3.10/site-packages/multiprocess/pool.py", line 48, in mapstar
    return list(map(*args))
  File "/home/acrnemr/miniconda3/envs/era5cl/lib/python3.10/site-packages/pathos/helpers/mp_helper.py", line 15, in <lambda>
    func = lambda args: f(*args)
  File "/home/acrnemr/miniconda3/envs/era5cl/lib/python3.10/site-packages/era5cli/fetch.py", line 441, in _getdata
    connection.retrieve(name, request, outputfile)
  File "/home/acrnemr/miniconda3/envs/era5cl/lib/python3.10/site-packages/cdsapi/api.py", line 348, in retrieve
    result = self._api("%s/resources/%s" % (self.url, name), request, "POST")
  File "/home/acrnemr/miniconda3/envs/era5cl/lib/python3.10/site-packages/cdsapi/api.py", line 505, in _api
    raise Exception(
Exception: no data is available within your requested subset. Request returned no data.

Discussion:
codecov[bot]: # [Codecov](https://codecov.io/gh/eWaterCycle/era5cli/pull/131?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=eWaterCycle) Report
> :exclamation: No coverage uploaded for pull request base (`main@2b8e449`). [Click here to learn what that means](https://docs.codecov.io/docs/error-reference?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=eWaterCycle#section-missing-base-commit).
> The diff coverage is `n/a`.

<details><summary>Additional details and impacted files</summary>


[![Impacted file tree graph](https://codecov.io/gh/eWaterCycle/era5cli/pull/131/graphs/tree.svg?width=650&height=150&src=pr&token=qeZXgGASBK&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=eWaterCycle)](https://codecov.io/gh/eWaterCycle/era5cli/pull/131?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=eWaterCycle)

```diff
@@           Coverage Diff           @@
##             main     #131   +/-   ##
=======================================
  Coverage        ?   98.95%           
=======================================
  Files           ?        6           
  Lines           ?      381           
  Branches        ?        0           
=======================================
  Hits            ?      377           
  Misses          ?        4           
  Partials        ?        0           
```



------

[Continue to review full report at Codecov](https://codecov.io/gh/eWaterCycle/era5cli/pull/131?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=eWaterCycle).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=eWaterCycle)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/eWaterCycle/era5cli/pull/131?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=eWaterCycle). Last update [2b8e449...b8b1d54](https://codecov.io/gh/eWaterCycle/era5cli/pull/131?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=eWaterCycle). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=eWaterCycle).

</details>
bvreede: @malininae Could you take a look at this PR? I can't seem to officially request your review, but since you raised the issue, I'd be very happy with your eyes on this fix! :)

Patch:
diff --git a/.zenodo.json b/.zenodo.json
index 8c63ee1..233162f 100644
--- a/.zenodo.json
+++ b/.zenodo.json
@@ -72,6 +72,11 @@
             "affiliation": "Netherlands eScience Center",
             "name": "Verhoeven, Stefan",
             "orcid": "0000-0002-5821-2060"
+        },
+        {
+            "affiliation": "Environment and Climate Change Canada",
+            "name": "Malinina, Elizaveta",
+            "orcid": "0000-0002-4102-2877"
         }
     ],
     "description": "A command line interface to download ERA5 data from the Climate Data Store.\n",
diff --git a/CITATION.cff b/CITATION.cff
index dc47990..1df530f 100644
--- a/CITATION.cff
+++ b/CITATION.cff
@@ -77,6 +77,11 @@ authors:
     family-names: Verhoeven
     given-names: Stefan
     orcid: https://orcid.org/0000-0002-5821-2060
+  -
+    affiliation: "Environment and Climate Change Canada"
+    family-names: Malinina
+    given-names: Elizaveta
+    orcid: https://orcid.org/0000-0002-4102-2877
 
 cff-version: 1.2.0
 date-released: 2021-11-30
diff --git a/era5cli/cli.py b/era5cli/cli.py
index f7cb0d7..13f6c23 100644
--- a/era5cli/cli.py
+++ b/era5cli/cli.py
@@ -148,7 +148,8 @@ def _build_parser():
                              Whether to download the preliminary back extension
                              (1950-1978). Note that when `--prelimbe` is used,
                              `--startyear` and `--endyear` should be set
-                             between 1950 and 1978.
+                             between 1950 and 1978. Please, be aware that
+                             ERA5 data is available from 1959.
                              `--prelimbe` is incompatible with `--land`.
 
                              ''')
@@ -159,7 +160,7 @@ def _build_parser():
         help=textwrap.dedent('''\
                              Whether to download data from the ERA5-Land
                              dataset. Note that the ERA5-Land dataset starts in
-                             1981.
+                             1950.
                              `--land` is incompatible with the use of
                              `--prelimbe` and `--ensemble`.
 
@@ -343,12 +344,12 @@ def _construct_year_list(args):
                 'year should be between 1950 and 1978'
             )
         elif args.land:
-            assert 1981 <= year <= datetime.now().year, (
-                'for ERA5-Land, year should be between 1981 and present'
+            assert 1950 <= year <= datetime.now().year, (
+                'for ERA5-Land, year should be between 1950 and present'
             )
         else:
-            assert 1979 <= year <= datetime.now().year, (
-                'year should be between 1979 and present'
+            assert 1959 <= year <= datetime.now().year, (
+                'year should be between 1959 and present'
             )
 
     assert endyear >= args.startyear, (
diff --git a/era5cli/fetch.py b/era5cli/fetch.py
index ad3883c..e623f9f 100644
--- a/era5cli/fetch.py
+++ b/era5cli/fetch.py
@@ -250,6 +250,10 @@ class Fetch:
 
     def _product_type(self):
         """Construct the product type name from the options."""
+        assert not (self.land and self.ensemble), (
+                'ERA5-Land does not contain Ensemble statistics.'
+            )
+
         if self.period == 'hourly' and self.ensemble and self.statistics:
             # The only configuration to return a list
             return [
@@ -387,7 +391,8 @@ class Fetch:
         if self.prelimbe:
             if self.land:
                 raise ValueError(
-                    "Back extension not (yet) available for ERA5-Land.")
+                    "Back extension not available for ERA5-Land. "
+                    "ERA5-Land data is available from 1950 on.")
             name += "-preliminary-back-extension"
         return name, variable
 


Test patch:
diff --git a/tests/test_cli.py b/tests/test_cli.py
index 1184ed1..e4e5155 100644
--- a/tests/test_cli.py
+++ b/tests/test_cli.py
@@ -138,7 +138,7 @@ def test_main_fetch(fetch):
         assert cli._execute(args)
 
     # should give an AssertionError if years are out of bounds
-    argv = ['hourly', '--startyear', '1950',
+    argv = ['hourly', '--startyear', '1949',
             '--variables', 'total_precipitation', '--statistics',
             '--endyear', '2007', '--ensemble']
     args = cli._parse_args(argv)
@@ -160,14 +160,6 @@ def test_main_fetch(fetch):
     args = cli._parse_args(argv)
     cli._execute(args)
 
-    # no land available for back extension
-    argv = ['monthly', '--startyear', '1980', '--endyear', '1980',
-            '--variables', 'total_precipitation', '--synoptic',
-            '--ensemble', '--land']
-    args = cli._parse_args(argv)
-    with pytest.raises(AssertionError):
-        cli._execute(args)
-
 
 @mock.patch("era5cli.info.Info", autospec=True)
 def test_main_info(info):
diff --git a/tests/test_fetch.py b/tests/test_fetch.py
index f35f2d4..7187c62 100644
--- a/tests/test_fetch.py
+++ b/tests/test_fetch.py
@@ -334,6 +334,10 @@ def test_product_type():
     producttype = era5._product_type()
     assert producttype is None
 
+    era5.ensemble = True
+    with pytest.raises(AssertionError):
+        producttype = era5._product_type()
+
 
 def test_check_levels():
     """Test _check_levels function of Fetch class"""