Repository: gammapy/gammapy

Issue:
Dataset energy range handling
All `Dataset` classes define different energy ranges related to the definition of the different `mask` attributes. Currently a single energy range is derived even if the energy range varies with e.g. position. This issue outlines a possible solution to make the handling uniform and more general. First I propose to define three energy ranges, related to the the masks defined on the dataset:
- `.energy_range`, which is the global energy range derived from `.mask`
- `.energy_range_safe`, which is the energy range derived from `.mask_safe`
- `.energy_range_fit`, which is the global energy range derived from `.mask_fit`
Instead of a single value for the min and max energy we should return a `Map` object to keep the information on the position dependency. E.g. it can be useful to plot a map of the lower energy threshold of a flux map.

In addition we should mirror the API on the `Datasets` object. The challenge here is that the datasets can have different types and geometries. So it requires some handling of the energy ranges between the different datasets.

Finally it could be very useful to store the information one the energy ranges used for e.g. flux map and flux points estimation on the corresponding result objects. The logic here is to reduce the energy range maps on the input datasets, e.g.. by taking the min or max energies in each dataset / bin group.  


Discussion:
Hi all,
There is a fourth type of energy range, I think, the (future) one coming from the IRFs... This range is the first that is defined in the data flow.
Thanks @bkhelifi indeed...in some cases it's equivalent to `energy_safe` but it doesn't have to be, depending on the choice of the user. On the other hand the recommended IRF energy range might not be the one actually used during the analysis, e.g. if a more strict energy range is applied by other criteria. So A good way might be handle the IRF energy range as part of `meta` information, because it is not necessarily equivalent to the energy range used in the analysis.
This is for the moment premature, as the IRFs does not contain energy ranges... We do not know how they will be stored. I do expect however that the maximal energy of validity of IRFs will depend on the offset (because at least of statistical issue). So at least an array will be needed, not only a meta information into the FITS header.
It is not needed to implement such `maps' of validity range of IRFs now. But it enters perfectly in the scheme that you are proposing...
@adonath Does linking #3020 mean that returning a `Map` object is not desired anymore? Because `_energy_range` could be expanded to support this feature independent of that PR.
Thanks a lot @alexiscoutinho for responding and sorry for the delay on #3020, it just was not on our priority list, until now. After some thought and discussion in the developer calls we agree that the most flexible solution would be indeed to return a `Map` object. In practice this means:

- Remove the `region` argument from `energy_range_` attributes
- Return a tuple of `Map` objects in all the `energy_range_` attributes 
- Adapt the implementation, such that the returned `Map` object contains the energy threshold, that varies with position

What do you think?
I tried to imply in my previous comment that `_energy_range` could return either a `Quantity` or `Map` object if `region` is passed or not, respectively. However, if it is decided that getting the greatest energy range in a certain region is not really useful (I don't know), thus justifying removing `region`, then I think closing #3020 would be the ideal action since it would be too different from its original proposition. No hard feelings either way.
Thanks a lot @alexiscoutinho! Getting the greatest energy in a range is still useful, however in the proposed scheme it can be achieved like so:
```python3
e_min, e_max = dataset.energy_range
e_max.to_region_nd_map(region, func=np.max)
```
The advantage is that more information is conserved in the `.energy_range` attribute. I'd be happy to review any new PR on this. If you prefer that we close #3020, that's fine by me as well...

Resolving patch:
diff --git a/gammapy/datasets/map.py b/gammapy/datasets/map.py
--- a/gammapy/datasets/map.py
+++ b/gammapy/datasets/map.py
@@ -337,34 +337,50 @@ def data_shape(self):
         """Shape of the counts or background data (tuple)"""
         return self._geom.data_shape
 
-    def _energy_range(self, mask = None):
-        """Compute the energy range with or without the mask fit."""
-        energy = self._geom.axes["energy"].edges
-        energy_min, energy_max = energy[:-1], energy[1:]
-
-        if mask is not None:
-            if mask.data.any():
-                mask = mask.data.any(axis=(1, 2))
+    def _energy_range(self, mask_map=None):
+        """Compute the energy range maps with or without the fit mask."""
+        geom = self._geom
+        energy = geom.axes["energy"].edges
+        e_i = geom.axes.index_data("energy")
+        geom = geom.drop("energy")
+
+        if mask_map is not None:
+            mask = mask_map.data
+            if mask.any():
+                idx = mask.argmax(e_i)
+                energy_min = energy.value[idx]
+                mask_nan = ~mask.any(e_i)
+                energy_min[mask_nan] = np.nan
+
+                mask = np.flip(mask, e_i)
+                idx = mask.argmax(e_i)
+                energy_max = energy.value[::-1][idx]
+                energy_max[mask_nan] = np.nan
             else:
-                return None, None
+                energy_min = np.full(geom.data_shape, np.nan)
+                energy_max = energy_min.copy()
         else:
-            mask = None
+            data_shape = geom.data_shape
+            energy_min = np.full(data_shape, energy.value[0])
+            energy_max = np.full(data_shape, energy.value[-1])
 
-        return u.Quantity([energy_min[mask].min(), energy_max[mask].max()])
+        map_min = Map.from_geom(geom, data=energy_min, unit=energy.unit)
+        map_max = Map.from_geom(geom, data=energy_max, unit=energy.unit)
+        return map_min, map_max
 
     @property
     def energy_range(self):
-        """Energy range defined by the full mask (mask_safe and mask_fit)."""
-        return self._energy_range(mask=self.mask)
+        """Energy range maps defined by the full mask (mask_safe and mask_fit)."""
+        return self._energy_range(self.mask)
 
     @property
     def energy_range_safe(self):
-        """Energy range defined by the mask_safe only."""
+        """Energy range maps defined by the mask_safe only."""
         return self._energy_range(self.mask_safe)
 
     @property
-    def energy_range_safe(self):
-        """Energy range defined by the mask_fit only."""
+    def energy_range_fit(self):
+        """Energy range maps defined by the mask_fit only."""
         return self._energy_range(self.mask_fit)
 
     def npred(self):
@@ -647,7 +663,7 @@ def stack(self, other, nan_to_num=True):
         ----------
         other: `~gammapy.datasets.MapDataset` or `~gammapy.datasets.MapDatasetOnOff`
             Map dataset to be stacked with this one. If other is an on-off
-            dataset alpha * counts_off is used as a background model.        
+            dataset alpha * counts_off is used as a background model.
         nan_to_num: bool
             Non-finite values are replaced by zero if True (default).
 
@@ -1734,7 +1750,7 @@ class MapDatasetOnOff(MapDataset):
 
     See Also
     --------
-    MapDatasetOn, SpectrumDataset, FluxPointsDataset
+    MapDataset, SpectrumDataset, FluxPointsDataset
 
     """
 


Test patch:
diff --git a/gammapy/datasets/tests/test_map.py b/gammapy/datasets/tests/test_map.py
--- a/gammapy/datasets/tests/test_map.py
+++ b/gammapy/datasets/tests/test_map.py
@@ -85,6 +85,19 @@ def geom():
     )
 
 
+@pytest.fixture
+def geom1():
+    e_axis = MapAxis.from_energy_bounds("0.1 TeV", "10 TeV", nbin=20)
+    t_axis = MapAxis.from_bounds(0, 10, 2, name="time", unit="s")
+    return WcsGeom.create(
+        skydir=(266.40498829, -28.93617776),
+        binsz=0.02,
+        width=(3, 2),
+        frame="icrs",
+        axes=[e_axis, t_axis],
+    )
+
+
 @pytest.fixture
 def geom_etrue():
     axis = MapAxis.from_energy_bounds("0.1 TeV", "10 TeV", nbin=3, name="energy_true")
@@ -170,7 +183,7 @@ def sky_model():
 
 
 def get_map_dataset(geom, geom_etrue, edisp="edispmap", name="test", **kwargs):
-    """Returns a MapDatasets"""
+    """Returns a MapDataset"""
     # define background model
     background = Map.from_geom(geom)
     background.data += 0.2
@@ -325,6 +338,50 @@ def test_to_spectrum_dataset(sky_model, geom, geom_etrue, edisp_mode):
     assert_allclose(spectrum_dataset_corrected.exposure.data[1], 2.05201e09, rtol=1e-5)
 
 
+@requires_data()
+def test_energy_range(sky_model, geom1, geom_etrue):
+    sky_coord1 = SkyCoord(266.5, -29.3, unit="deg")
+    region1 = CircleSkyRegion(sky_coord1, 0.5 * u.deg)
+    mask1 = geom1.region_mask([region1]) & geom1.energy_mask(1 * u.TeV, 7 * u.TeV)
+    sky_coord2 = SkyCoord(266.5, -28.7, unit="deg")
+    region2 = CircleSkyRegion(sky_coord2, 0.5 * u.deg)
+    mask2 = geom1.region_mask([region2]) & geom1.energy_mask(2 * u.TeV, 8 * u.TeV)
+    mask3 = geom1.energy_mask(3 * u.TeV, 6 * u.TeV)
+
+    mask_safe = Map.from_geom(geom1, data=(mask1 | mask2 | mask3).data)
+    dataset = get_map_dataset(geom1, geom_etrue, edisp=None, mask_safe=mask_safe)
+    energy = geom1.axes["energy"].edges.value
+
+    e_min, e_max = dataset.energy_range_safe
+    assert_allclose(e_min.get_by_coord((265, -28, 0)), energy[15])
+    assert_allclose(e_max.get_by_coord((265, -28, 5)), energy[17])
+    assert_allclose(e_min.get_by_coord((sky_coord1.ra, sky_coord1.dec, 6)), energy[10])
+    assert_allclose(e_max.get_by_coord((sky_coord1.ra, sky_coord1.dec, 1)), energy[18])
+    assert_allclose(e_min.get_by_coord((sky_coord2.ra, sky_coord2.dec, 2)), energy[14])
+    assert_allclose(e_max.get_by_coord((sky_coord2.ra, sky_coord2.dec, 7)), energy[19])
+    assert_allclose(e_min.get_by_coord((266.5, -29, 8)), energy[10])
+    assert_allclose(e_max.get_by_coord((266.5, -29, 3)), energy[19])
+
+    e_min, e_max = dataset.energy_range_fit
+    assert_allclose(e_min.get_by_coord((265, -28, 0)), np.nan)
+    assert_allclose(e_max.get_by_coord((265, -28, 5)), np.nan)
+    assert_allclose(e_min.get_by_coord((266, -29, 4)), energy[0])
+    assert_allclose(e_max.get_by_coord((266, -29, 9)), energy[20])
+
+    e_min, e_max = dataset.energy_range
+    assert_allclose(e_min.get_by_coord((266, -29, 4)), energy[15])
+    assert_allclose(e_max.get_by_coord((266, -29, 9)), energy[17])
+
+    mask_zeros = Map.from_geom(geom1, data=np.zeros_like(mask_safe))
+    e_min, e_max = dataset._energy_range(mask_zeros)
+    assert_allclose(e_min.get_by_coord((266.5, -29, 8)), np.nan)
+    assert_allclose(e_max.get_by_coord((266.5, -29, 3)), np.nan)
+
+    e_min, e_max = dataset._energy_range()
+    assert_allclose(e_min.get_by_coord((265, -28, 0)), energy[0])
+    assert_allclose(e_max.get_by_coord((265, -28, 5)), energy[20])
+
+
 @requires_data()
 def test_info_dict(sky_model, geom, geom_etrue):
     dataset = get_map_dataset(geom, geom_etrue)
diff --git a/gammapy/datasets/tests/test_spectrum.py b/gammapy/datasets/tests/test_spectrum.py
--- a/gammapy/datasets/tests/test_spectrum.py
+++ b/gammapy/datasets/tests/test_spectrum.py
@@ -75,9 +75,11 @@ def test_str(spectrum_dataset):
 
 
 def test_energy_range(spectrum_dataset):
-    energy_range = spectrum_dataset.energy_range
-    assert energy_range.unit == u.TeV
-    assert_allclose(energy_range.to_value("TeV"), [0.1, 10.0])
+    e_min, e_max = spectrum_dataset.energy_range
+    assert e_min.unit == u.TeV
+    assert e_max.unit == u.TeV
+    assert_allclose(e_min, 0.1)
+    assert_allclose(e_max, 10.0)
 
 
 def test_info_dict(spectrum_dataset):
@@ -181,7 +183,7 @@ def test_spectrum_dataset_create():
     assert empty_spectrum_dataset.edisp.edisp_map.geom.axes["energy"].nbin == 2
     assert empty_spectrum_dataset.gti.time_sum.value == 0
     assert len(empty_spectrum_dataset.gti.table) == 0
-    assert empty_spectrum_dataset.energy_range[0] is None
+    assert np.isnan(empty_spectrum_dataset.energy_range[0])
     assert_allclose(empty_spectrum_dataset.mask_safe, 0)
 
 
@@ -438,7 +440,7 @@ def test_spectrum_dataset_on_off_create(self):
         assert empty_dataset.acceptance_off.data.shape[0] == 2
         assert empty_dataset.gti.time_sum.value == 0
         assert len(empty_dataset.gti.table) == 0
-        assert empty_dataset.energy_range[0] is None
+        assert np.isnan(empty_dataset.energy_range[0])
 
     def test_create_stack(self):
         geom = RegionGeom(region=None, axes=[self.e_reco])
@@ -447,7 +449,10 @@ def test_create_stack(self):
         stacked.mask_safe.data += True
 
         stacked.stack(self.dataset)
-        assert_allclose(stacked.energy_range.value, self.dataset.energy_range.value)
+        e_min_stacked, e_max_stacked = stacked.energy_range
+        e_min_dataset, e_max_dataset = self.dataset.energy_range
+        assert_allclose(e_min_stacked, e_min_dataset)
+        assert_allclose(e_max_stacked, e_max_dataset)
 
     def test_alpha(self):
         assert self.dataset.alpha.data.shape == (4, 1, 1)
@@ -764,7 +769,7 @@ def test_fit_range(self):
         actual = obs.energy_range[0]
 
         assert actual.unit == "keV"
-        assert_allclose(actual.value, 8.912509e08)
+        assert_allclose(actual, 8.912509e08)
 
     def test_no_edisp(self):
         dataset = self.datasets[0].copy()
@@ -910,10 +915,10 @@ def test_thresholds(self):
         energy_min, energy_max = self.stacked_dataset.energy_range
 
         assert energy_min.unit == "keV"
-        assert_allclose(energy_min.value, 8.912509e08, rtol=1e-3)
+        assert_allclose(energy_min, 8.912509e08, rtol=1e-3)
 
         assert energy_max.unit == "keV"
-        assert_allclose(energy_max.value, 4.466836e10, rtol=1e-3)
+        assert_allclose(energy_max, 4.466836e10, rtol=1e-3)
 
     def test_verify_npred(self):
         """Verifying npred is preserved during the stacking"""
@@ -1135,8 +1140,8 @@ def test_fit_range(self):
         assert np.sum(dataset.mask_safe) == self.nbins
         energy_min, energy_max = dataset.energy_range
 
-        assert_allclose(energy_max.value, 10)
-        assert_allclose(energy_min.value, 0.1)
+        assert_allclose(energy_max, 10)
+        assert_allclose(energy_min, 0.1)
 
     def test_stat_profile(self):
         geom = self.src.geom
diff --git a/gammapy/makers/tests/test_spectrum.py b/gammapy/makers/tests/test_spectrum.py
--- a/gammapy/makers/tests/test_spectrum.py
+++ b/gammapy/makers/tests/test_spectrum.py
@@ -173,7 +173,7 @@ def test_safe_mask_maker_dl3(spectrum_dataset_crab, observations_hess_dl3):
     obs = observations_hess_dl3[0]
     dataset = maker.run(spectrum_dataset_crab, obs)
     dataset = safe_mask_maker.run(dataset, obs)
-    assert_allclose(dataset.energy_range[0].value, 1)
+    assert_allclose(dataset.energy_range[0], 1)
     assert dataset.energy_range[0].unit == "TeV"
 
     mask_safe = safe_mask_maker.make_mask_energy_aeff_max(dataset)
@@ -194,7 +194,7 @@ def test_safe_mask_maker_dc1(spectrum_dataset_gc, observations_cta_dc1):
     maker = SpectrumDatasetMaker()
     dataset = maker.run(spectrum_dataset_gc, obs)
     dataset = safe_mask_maker.run(dataset, obs)
-    assert_allclose(dataset.energy_range[0].value, 3.162278, rtol=1e-3)
+    assert_allclose(dataset.energy_range[0], 3.162278, rtol=1e-3)
     assert dataset.energy_range[0].unit == "TeV"