Repository: gammapy/gammapy

Issue:
3D (Fermi) + 1D (LST/MAGIC PointSkyRegion-style) analysis. Global fit ignores the 1D. 
**Gammapy version**
0.20.1

Are you using the latest stable version of Gammapy? (see https://gammapy.org/news.html)
yes

**Bug description**
Issue discussed already in slack (and I should probably have opened this ticket back then, my bad). A 3D (e.g. Fermi) and a 1D ON/OFF datasets (PointSkyRegion type, e.g. LST or MAGIC) are analyzed together with N skymodels for the 3D (with spatial and spectral components, including those for the source of interest) and 1 skymodel (the same model as in the 3D for ONLY the source of interest, including spectral and spatial components). The result, after the likelihood fit, is that one of the datasets is completely ignored in the fit. However, no error or warning is raised, so I have no idea how to debug it. 

<img width="730" alt="Captura de Pantalla 2022-09-29 a las 11 35 47" src="https://user-images.githubusercontent.com/2607760/193009687-d5d1bee3-9a8e-4fa0-852e-d625918a63cd.png">

Partial workaround: 
1) define a skymodel for the 1D dataset consisting only on a spectral component, no spatial ones. 
2) The parameters of the spectral components are linked/fixed to the corresponding spectral component of the source model in the 3D dataset (lets call the source name in the 3D dataset XXX). 
3) The source name of the skymodel then needs to be different, e.g. renamed to XXX-2, otherwise gammapy returns an error saying that the names should be unique.
4) perform the fit, and then voila both datasets contribute to the result, the confidence band is much narrower.

<img width="738" alt="Captura de Pantalla 2022-09-29 a las 11 35 57" src="https://user-images.githubusercontent.com/2607760/193009783-7fc169bf-0e19-4e89-887f-4b8cd6335b7f.png">

However, this is not a perfect solution, because the FluxPointEstimator needs to be called for each dataset with their unique source names (even when it is the same source and both datasets should contribute to the energy ranges where both instruments are sensitive). So you have Fermi points on one hand and IACT points on the other. At the overlapping energy range (~20-300 GeV)  you may have 'duplicated' points with larger error bars (because only one dataset contributes at a time). 

**Expected behavior**

The first option, i.e. defining 'datasets' to include both the 3D and the 1D (PointSkyRegion) and then making the fit should work fine, with the only difference with respect to a 3D that for the 1D model the spatial component should not contribute to the likelihood. Or if there is any reason why it shouldn't, it should at least raise an error instead of silently ignoring one of the datasets.

Another possible solution would be to try to get the FluxPointEstimation to also use the two datasets to compute each flux point, but that would involve dealing with two datasets with a source with different names in each. If the global fit works this way, no reason why the flux point estimation proceedure, which basically re-optimizes the 'norm' in each bin, shouldn't. But it sounds more 'hacky' to me than just fixing the first option and allowing the minimizer to work with this 1D+3D datasets

**To Reproduce**
1) Build a 3D dataset (suggested from a Fermi analysis done with the Fermitools).
2) Build a 1D dataset (energy-dependent / PointSkyRegion style, I ignore if the standard CircleSkyRegion analysis reproduces the issue)
3) Assign all the spatial+spectral skymodels for each source in the field of the 3D dataset.
4) Assign the spatial+spectral for the source of interest to match the one in the 3D dataset. 
5) Build the datasets([1D,3D]) object.
6) Make the likelihood fit with the models

**Other information**
Any other information you think will be useful for us to fix the issue can go here.



Discussion:
Thanks @mireianievas! I looked into this issue a while ago and found it was essentially related to the behavior of `PointSkyRegion.contains()` (https://github.com/astropy/regions/issues/474). Have you checked that the issue is still present in the current dev version?
You mean the dev version of astropy? @adonath 

I will try asap :) 
In [regions/shapes/point.py](https://github.com/astropy/regions/blob/main/regions/shapes/point.py) I see the following bit of code

```
    def contains(self, pixcoord):
        if pixcoord.isscalar:
            in_reg = False
        else:
            in_reg = np.zeros(pixcoord.x.shape, dtype=bool)

        if self.meta.get('include', True):
            # in_reg = False, always.  Points do not include anything.
            return in_reg
        else:
            return np.logical_not(in_reg)
```

Knowing it is a bit of workaround, perhaps I could 'trick' that call setting that meta = {'include' = False} ?

Or just mock patch the function/module. 

Amazing, the trick seems to work @adonath . All I had to do was setting my

on_pointskyregion.meta = {'include': False}

and then the PointSkyRegion returns the opposite boolean ... True. 

dataset.npred() for the 1D dataset also changed:
 
```
array([0.0000e+00, 0.0000e+00, 0.0000e+00, 0.0000e+00, 0.0000e+00,
       0.0000e+00, 0.0000e+00, 0.0000e+00, 2.3745e+03, 1.4935e+03,
       6.4250e+02, 3.4000e+02, 1.0000e+02, 5.7500e+01, 3.4000e+01,
       1.8000e+01, 1.2500e+01, 4.5000e+00, 4.0000e+00, 0.0000e+00,
       1.0000e+00, 0.0000e+00, 0.0000e+00, 0.0000e+00])
```

to 

```
array([0.00000000e+00, 0.00000000e+00, 0.00000000e+00, 0.00000000e+00,
       0.00000000e+00, 0.00000000e+00, 0.00000000e+00, 0.00000000e+00,
       2.44549339e+03, 1.55626963e+03, 7.01364692e+02, 3.77371018e+02,
       1.29246339e+02, 7.18087605e+01, 4.28105210e+01, 2.38393481e+01,
       1.53224761e+01, 6.62009372e+00, 4.39779027e+00, 1.04601812e+00,
       1.48030733e+00, 1.95032540e-01, 5.64593577e-02, 6.86080656e-03])
```
![image](https://user-images.githubusercontent.com/2607760/193270731-9dc72af5-e48e-4a75-9ad6-c38029cec7b1.png)

I got my hopes too high. The global fit now works, but the FluxPointEstimator still doesn't

```
flux_points = fpe.run(datasets=datasets)

-----------

File ~/anaconda/envs/gammapy-dev/lib/python3.8/site-packages/gammapy/estimators/points/sed.py:117, in FluxPointsEstimator.run(self, datasets)
    115 table = table_from_row_data(rows=rows, meta=meta)
    116 model = datasets.models[self.source]
--> 117 return FluxPoints.from_table(
    118     table=table,
    119     reference_model=model.copy(),
    120     gti=datasets.gti,
    121     format="gadf-sed",
    122 )

File ~/anaconda/envs/gammapy-dev/lib/python3.8/site-packages/gammapy/estimators/points/core.py:249, in FluxPoints.from_table(cls, table, sed_type, format, reference_model, gti)
    247 for name in cls.all_quantities(sed_type=sed_type):
    248     if name in table.colnames:
--> 249         maps[name] = RegionNDMap.from_table(
    250             table=table, colname=name, format=format
    251         )
    253 meta = cls._get_meta_gadf(table)
    254 return cls.from_maps(
    255     maps=maps,
    256     reference_model=reference_model,
   (...)
    259     gti=gti,
    260 )

File ~/anaconda/envs/gammapy-dev/lib/python3.8/site-packages/gammapy/maps/region/ndmap.py:531, in RegionNDMap.from_table(cls, table, format, colname)
    528 if colname is None:
    529     raise ValueError("Column name required")
--> 531 axes = MapAxes.from_table(table=table, format=format)
    533 if colname == "stat_scan":
    534     names = ["norm", "energy"]

File ~/anaconda/envs/gammapy-dev/lib/python3.8/site-packages/gammapy/maps/axes.py:1991, in MapAxes.from_table(cls, table, format)
   1989 for axis_format in ["gadf-sed-norm", "gadf-sed-energy", "gadf-sed-counts"]:
   1990     try:
-> 1991         axis = MapAxis.from_table(table=table, format=axis_format)
   1992     except KeyError:
   1993         continue

File ~/anaconda/envs/gammapy-dev/lib/python3.8/site-packages/gammapy/maps/axes.py:1320, in MapAxis.from_table(cls, table, format, idx, column_prefix)
   1318 if "datasets" in table.colnames:
   1319     labels = np.unique(table["datasets"])
-> 1320     axis = LabelMapAxis(labels=labels, name="dataset")
   1321 else:
   1322     shape = table["counts"].shape

File ~/anaconda/envs/gammapy-dev/lib/python3.8/site-packages/gammapy/maps/axes.py:2763, in LabelMapAxis.__init__(self, labels, name)
   2762 def __init__(self, labels, name=""):
-> 2763     unique_labels = set(labels)
   2765     if not len(unique_labels) == len(labels):
   2766         raise ValueError("Node labels must be unique")

TypeError: unhashable type: 'list'
```
@mireianievas It seems you use a list a dataset name somewhere in your code, can you check?
Yes, setting the meta on the regions works, but only accidentally. I'll check whether we still need a small fix in Gammapy...
@mireianievas Can you please take a brief look at #4118?
I don't think so

```
Datasets
--------

Dataset 0: 

  Type       : MapDataset
  Name       : Fermi-LAT_FRONT
  Instrument : 
  Models     : [ 'diffuse-iem', 'fermi-diffuse-iso-FRONT', 'MySource']

Dataset 1: 

  Type       : SpectrumDatasetOnOff
  Name       : MAGIC
  Instrument : 
  Models     : ['MySource']

```

This is what I pass to the ```FluxPointsEstimator(...).run()```
Thanks @mireianievas, this looks all good! I'll check once more...
@mireianievas Which version of Astropy do you use? What is the output of `gammapy info`?

```
Gammapy package:

	version                : 0.20.1     
	path                   : /home/mnievas/anaconda/envs/gammapy-dev/lib/python3.8/site-packages/gammapy 


Other packages:

	numpy                  : 1.23.1     
	scipy                  : 1.8.1      
	astropy                : 5.1        
	regions                : 0.6        
	click                  : 8.1.3      
	yaml                   : 6.0        
	IPython                : 8.4.0      
	jupyterlab             : 3.4.4      
	matplotlib             : 3.5.2      
	pandas                 : 1.4.3      
	healpy                 : 1.16.1     
	iminuit                : 2.15.0     
	sherpa                 : 4.14.1     
	naima                  : 0.10.0     
	emcee                  : 3.1.2      
	corner                 : 2.2.1 
```
I think it is because the Datasets names in one of the models (the one for my source of interest) is indeed a list. 

```
Component XX: SkyModel

  Name                      : XXX
  Datasets names            : ['Fermi-LAT_FRONT', 'MAGIC']
  Spectral model type       : CompoundSpectralModel
  Spatial  model type       : PointSpatialModel
  Temporal model type       : 
  Parameters:
    amplitude                     :   1.70e-08   +/- 3.4e-09 1 / (cm2 s TeV)
    reference             (frozen):     10.000       GeV         
    alpha                         :      1.955   +/-    0.04             
    beta                          :      0.020   +/-    0.01             
    alpha_norm            (frozen):      1.000                   
    redshift              (frozen):      0.100                   
    lon_0                 (frozen):    XXX      deg         
    lat_0                 (frozen):     XXX       deg   
```
I found this

```
class LabelMapAxis:
    """Map axis using labels

    Parameters
    ----------
    labels : list of str
        Labels to be used for the axis nodes.
    name : str
        Name of the axis.

    """

    node_type = "label"

    def __init__(self, labels, name=""):
        unique_labels = set(labels)
        if not len(unique_labels) == len(labels):
            raise ValueError("Node labels must be unique")

        self._labels = np.array(labels)
        self._name = name
```

why do we convert from a list of str into a set? just to get the unique ones? why not doing a unique_labels = np.unique(labels) and then we do not need to do the conversion into np.array() ? 
@mireianievas I have looked into the issue and it is related to the fact that the datasets cover a different energy range. So there are flux points where only one of each dataset contributes, and there are points where both contribute.

This is currently not handled by Gammapy...I'll check if there is a workaround. 
![image](https://user-images.githubusercontent.com/2607760/193324774-4a3b2d35-268e-4977-97f4-315f0dc1fd09.png)

I had to make a few changes in some places though:

Adding in gammapy/maps/maps.py

```
def __setitem__(self, key, value):
        if value is not None and not isinstance(value, Map) and **not isinstance(value,RegionNDMap):**
```

In gammapy/maps/axes.py replace set(labels) by np.unique(labels). Then self._labels = unique_labels (no need to make a copy as a np.array anymore.

```
   def __init__(self, labels, name=""):
        print(labels)
        #unique_labels = set(labels)
        unique_labels = np.unique(labels)

        if not len(unique_labels) == len(labels):
            raise ValueError("Node labels must be unique")

        self._labels = unique_labels #np.array(labels)
        self._name = name
```

@mireianievas I'm proposing a different fix in #4118. But I also like the simplification introduced by using `np.unique`, feel free to open a PR and contribute this. 

I'm less convinced about the change in `__setitem__`. `RegonNDMap` inherits from `Map` so the statement should have no effect...
@adonath does #4118 already include fixes for the SED points calculation? if so I can test it. 

I introduced the RegionNDMap change because, despite inherting Map, it enters that if statement for some reason. 
By the way, shouldn't this be an element-wise comparison?

```
changed = self._cached_parameter_values_previous == values
```

in gammapy/datasets/evaluator.py:404

```
    def parameter_norm_only_changed(self):
        """Only norm parameter changed"""
        norm_only_changed = False
        idx = self._norm_idx
        values = self.model.parameters.value
        if idx and self._computation_cache is not None:
            changed = self._cached_parameter_values_previous == values
            norm_only_changed = sum(changed) == 1 and changed[idx]
        if not norm_only_changed:
            self._cached_parameter_values_previous = values
        return norm_only_changed
```
> By the way, shouldn't this be an element-wise comparison?

It is a Numpy array which is compared element-wise. I don't see any issue here.

In #4118 I added the fix for the first issue. The support for datasets of multiple instruments requires and adaption of the current methods to be well defined. I opened a reminder issue to support this in #4123.
Sorry to rebump this. 

It should be fixed in the main repository, right?

I synced it yesterday and it does not look like it is fixed yet

![image](https://user-images.githubusercontent.com/2607760/195060221-a23402ff-ec82-49d0-bb2c-adef7ac2c53a.png)

That is a Fermi-LAT + LST analysis. LST analysis alone is pretty constraining, so that really is not showing up in the fitted spectrum. 

```
>> git rev-parse --abbrev-ref HEAD
main
```

I synced it yesterday (last commit I have is from Oct 7).




@mireianievas Yes, the specific fix for this issue is included. However it still requires that the center of the point region and the position of the model basically **exactly** agree. Is this the case? 
@registerrier I think we should maybe still add a note in our docs that in the joint fit scenario with a `PointSourceRegion` and varying rad max the position of the source cannot be fitted and should always be frozen. 
then maybe there should be a message, or log or something saying which datasets are used and which aren't. Right now it is completely silent I think, it is not straightforward to check if the data is being used at all. 

But I personally think this is not how it should be. In a Point-like / 1D analysis, at least at this late stage of the analysis, position accuracy is not that relevant. One knows which source is being observed and in this particular case, even if LST is managing internally a source region a few arcsecs off from that of LAT, that should make no difference and the summed likelihood should be allowed. 
Same fit with the old method (they should agree) of defining a model with only a spectral component for the LST data

![image](https://user-images.githubusercontent.com/2607760/195168279-eaa86aae-ab3e-462c-abbc-862d3fee54af.png)


The confidence band is super thin (even if the spectrum has quite high significance because it is a bright source, maybe the constraints are a bit too good). 

I made sure I used the same coordinates to define the ON region in my LST dataset analysis. And it returns True when I call manually the contains() function from my Jupyter notebook using that ON region. So I don't know what can be wrong, but for now it does not seem like the fix works in this particular case. 


Thanks @mireianievas for checking again! I'll investigate once more...
Just confirm @mireianievas , the issue was not correctly addressed by #4118. I have opened a new PR in #4133. DO you have time to check and confirm the fix? Thanks!
let me check it @adonath , thanks for the pull request !.
@mireianievas Consistent with the behavior for e.g. `MapDataset` with a `WcsGeom` Gammapy allows for a shift in position by `0.1 deg`, defined by https://github.com/gammapy/gammapy/blob/main/gammapy/datasets/evaluator.py#L13.
Still running, it is painfully slow to fit. 
No, it does not seem to give the expected results

![image](https://user-images.githubusercontent.com/2607760/195196676-ea86f1bd-b2e3-400e-a7cb-2559f087c6a8.png)

@mireianievas have you checked out the branch with the fix? It is not merged yet...

@adonath yes, I checked out the pull request directly

```
git fetch -u origin pull/4133/head:main
pip install .
```

(and I checked manually that the _regions_contain and the other changes are there)
Just to make sure

![image](https://user-images.githubusercontent.com/2607760/195199546-b70e9db4-10ac-4d37-85fb-8227279434f5.png)


So yes, the notebook is using the correct PR. And also the contains() function returns True when I pass the position of the source in the Fermi dataset and the wcs that I store in the exclusion mask I generate for the LST analysis.

Resolving patch:
diff --git a/gammapy/datasets/map.py b/gammapy/datasets/map.py
--- a/gammapy/datasets/map.py
+++ b/gammapy/datasets/map.py
@@ -1357,11 +1357,12 @@ def info_dict(self, in_safe_data_range=True):
 
         counts = 0
         background, excess, sqrt_ts = np.nan, np.nan, np.nan
+
         if self.counts:
-            summed_stat = self._counts_statistic[mask].sum()
-            counts = summed_stat.n_on
+            counts = self.counts.data[mask].sum()
 
             if self.background:
+                summed_stat = self._counts_statistic[mask].sum()
                 background = summed_stat.n_bkg
                 excess = summed_stat.n_sig
                 sqrt_ts = summed_stat.sqrt_ts
diff --git a/gammapy/maps/region/geom.py b/gammapy/maps/region/geom.py
--- a/gammapy/maps/region/geom.py
+++ b/gammapy/maps/region/geom.py
@@ -17,7 +17,6 @@
     PixCoord,
     PointSkyRegion,
     RectanglePixelRegion,
-    RectangleSkyRegion,
     Regions,
     SkyRegion,
 )
@@ -208,6 +207,8 @@ def contains(self, coords):
         """Check if a given map coordinate is contained in the region.
         Requires the `.region` attribute to be set.
 
+        For `PointSkyRegion` the method always returns true.
+
         Parameters
         ----------
         coords : tuple, dict, `MapCoord` or `~astropy.coordinates.SkyCoord`
@@ -225,21 +226,16 @@ def contains(self, coords):
 
         coords = MapCoord.create(coords, frame=self.frame, axis_names=self.axes.names)
 
-        if isinstance(self.region, PointSkyRegion):
-            # the point region is approximated by 1x1 pixel here
-            region = RectangleSkyRegion(
-                center=self.center_skydir,
-                width=self.binsz_wcs[0],
-                height=self.binsz_wcs[1],
-            )
-        else:
-            region = self.region
+        if self.is_all_point_sky_regions:
+            return np.ones(coords.skycoord.shape, dtype=bool)
 
-        return region.contains(coords.skycoord, self.wcs)
+        return self.region.contains(coords.skycoord, self.wcs)
 
     def contains_wcs_pix(self, pix):
         """Check if a given wcs pixel coordinate is contained in the region.
 
+        For `PointSkyRegion` the method always returns true.
+
         Parameters
         ----------
         pix : tuple
@@ -250,6 +246,9 @@ def contains_wcs_pix(self, pix):
         containment : `~numpy.ndarray`
             Bool array.
         """
+        if self.is_all_point_sky_regions:
+            return np.ones(pix[0].shape, dtype=bool)
+
         region_pix = self.region.to_pixel(self.wcs)
         return region_pix.contains(PixCoord(pix[0], pix[1]))
 
@@ -540,14 +539,7 @@ def coord_to_pix(self, coords):
         if self.region is None:
             pix = (0, 0)
         else:
-            # TODO: remove once fix is available in regions
-            if isinstance(self.region, PointSkyRegion):
-                point_region = self.region.to_pixel(self.wcs)
-                point_region.meta["include"] = False
-                pix_coord = PixCoord.from_sky(coords.skycoord, self.wcs)
-                in_region = point_region.contains(pix_coord)
-            else:
-                in_region = self.region.contains(coords.skycoord, wcs=self.wcs)
+            in_region = self.contains(coords.skycoord)
 
             x = np.zeros(coords.skycoord.shape)
             x[~in_region] = np.nan
diff --git a/gammapy/modeling/models/cube.py b/gammapy/modeling/models/cube.py
--- a/gammapy/modeling/models/cube.py
+++ b/gammapy/modeling/models/cube.py
@@ -271,6 +271,9 @@ def contributes(self, mask, margin="0 deg"):
             mask = mask.reduce_over_axes(func=np.logical_or)
 
         if mask.geom.is_region and mask.geom.region is not None:
+            if mask.geom.is_all_point_sky_regions:
+                return True
+
             geom = mask.geom.to_wcs_geom()
             mask = geom.region_mask([mask.geom.region])
 


Test patch:
diff --git a/gammapy/makers/tests/test_reduce.py b/gammapy/makers/tests/test_reduce.py
--- a/gammapy/makers/tests/test_reduce.py
+++ b/gammapy/makers/tests/test_reduce.py
@@ -307,7 +307,7 @@ def test_dataset_maker_spectrum_global_rad_max():
     assert counts.unit == ""
     assert counts_off.unit == ""
     assert_allclose(counts.data.sum(), 438, rtol=1e-5)
-    assert_allclose(counts_off.data.sum(), 273, rtol=1e-5)
+    assert_allclose(counts_off.data.sum(), 276, rtol=1e-5)
 
 
 @requires_data()
diff --git a/gammapy/maps/region/tests/test_ndmap.py b/gammapy/maps/region/tests/test_ndmap.py
--- a/gammapy/maps/region/tests/test_ndmap.py
+++ b/gammapy/maps/region/tests/test_ndmap.py
@@ -32,6 +32,19 @@ def region_map():
     return m
 
 
+@pytest.fixture
+def region_map_no_region():
+    axis = MapAxis.from_energy_bounds("1 TeV", "10 TeV", nbin=6, name="energy")
+    m = Map.create(
+        region=None,
+        map_type="region",
+        axes=[axis],
+        unit="1/TeV",
+    )
+    m.data = np.arange(m.data.size, dtype=float).reshape(m.geom.data_shape)
+    return m
+
+
 @pytest.fixture
 def point_region_map():
     axis = MapAxis.from_energy_bounds("1 TeV", "10 TeV", nbin=6, name="energy")
@@ -211,10 +224,13 @@ def test_region_nd_map_get(region_map):
     values = region_map.get_by_coord((83.63, 21.51, energies[[0, -1]]))
     assert_allclose(values.squeeze(), [0, 5])
 
-    values = region_map.get_by_coord((energies[[0, -1]],))
+
+def test_region_nd_map_get_no_region(region_map_no_region):
+    energies = region_map_no_region.geom.axes[0].center
+    values = region_map_no_region.get_by_coord((energies[[0, -1]],))
     assert_allclose(values.squeeze(), [0, 5])
 
-    values = region_map.get_by_coord({"energy": energies[[0, -1]]})
+    values = region_map_no_region.get_by_coord({"energy": energies[[0, -1]]})
     assert_allclose(values.squeeze(), [0, 5])
 
 
diff --git a/gammapy/modeling/models/tests/test_cube.py b/gammapy/modeling/models/tests/test_cube.py
--- a/gammapy/modeling/models/tests/test_cube.py
+++ b/gammapy/modeling/models/tests/test_cube.py
@@ -11,7 +11,7 @@
 from gammapy.data.gti import GTI
 from gammapy.datasets.map import MapEvaluator
 from gammapy.irf import EDispKernel, PSFKernel
-from gammapy.maps import Map, MapAxis, RegionGeom, WcsGeom
+from gammapy.maps import Map, MapAxis, RegionGeom, RegionNDMap, WcsGeom
 from gammapy.modeling import Parameter
 from gammapy.modeling.models import (
     CompoundSpectralModel,
@@ -712,3 +712,11 @@ def test_compound_spectral_model(caplog):
         name="source-1",
     )
     assert_allclose(m.spectral_model(5 * u.TeV).value, 2.87e-12, rtol=1e-2)
+
+
+def test_sky_model_contributes_point_region():
+    model = SkyModel.create("pl", "point")
+
+    geom = RegionGeom.create("icrs;point(0.05, 0.05)", binsz_wcs="0.01 deg")
+    mask = RegionNDMap.from_geom(geom)
+    assert np.any(model.contributes(mask))