Repository: gammapy/gammapy

Issue:
`MapDataset.copy` messes up model information?
Suppose I have a dataset, `dataset_fit`. 

>`print(dataset_fit)`

"""
MapDataset
----------

  Name                            : dataset-fit 

  Total counts                    : 176154 
  Total predicted counts          : 167638.00
  Total background counts         : 161422.07

  Exposure min                    : 3.46e+07 m2 s
  Exposure max                    : 2.52e+10 m2 s

  Number of total bins            : 810000 
  Number of fit bins              : 775332 

  Fit statistic type              : cash
  Fit statistic value (-2 log(L)) : 542119.02

  Number of models                : 2 
  Number of parameters            : 11
  Number of free parameters       : 5

  Component 0: SkyModel
  
    Name                      : model-fit
    Datasets names            : None
    Spectral model type       : PowerLawSpectralModel
    Spatial  model type       : GaussianSpatialModel
    Temporal model type       : None
    Parameters:
      index                   :   2.711              
      amplitude               :   1.13e-11  1 / (cm2 s TeV)
      reference    (frozen)   :   1.000  TeV         
      lon_0                   :   0.971  deg         
      lat_0                   :   0.977  deg         
      sigma                   :   0.339  deg         
      e            (frozen)   :   0.000              
      phi          (frozen)   :   0.000  deg         
  
  Component 1: BackgroundModel
  
    Name                      : dataset-simu-bkg
    Datasets names            : ['dataset-fit']
    Parameters:
      norm         (frozen)   :   1.000              
      tilt         (frozen)   :   0.000              
      reference    (frozen)   :   1.000  TeV         
  """

now, when I make a copy of this dataset, the `npred` goes totally bizarre.

>`ds2 = dataset_fit.copy()`
>`print(ds2)`

"""
MapDataset
----------

  Name                            : mALwImny 

  Total counts                    : 176154 
  Total predicted counts          : 6215.94
  Total background counts         : 161422.07

  Exposure min                    : 3.46e+07 m2 s
  Exposure max                    : 2.52e+10 m2 s

  Number of total bins            : 810000 
  Number of fit bins              : 775332 

  Fit statistic type              : cash
  Fit statistic value (-2 log(L)) : 3564900.55

  Number of models                : 2 
  Number of parameters            : 11
  Number of free parameters       : 5

  Component 0: SkyModel
  
    Name                      : model-fit
    Datasets names            : None
    Spectral model type       : PowerLawSpectralModel
    Spatial  model type       : GaussianSpatialModel
    Temporal model type       : None
    Parameters:
      index                   :   2.711              
      amplitude               :   1.13e-11  1 / (cm2 s TeV)
      reference    (frozen)   :   1.000  TeV         
      lon_0                   :   0.971  deg         
      lat_0                   :   0.977  deg         
      sigma                   :   0.339  deg         
      e            (frozen)   :   0.000              
      phi          (frozen)   :   0.000  deg         
  
  Component 1: BackgroundModel
  
    Name                      : dataset-simu-bkg
    Datasets names            : ['dataset-fit']
    Parameters:
      norm         (frozen)   :   1.000              
      tilt         (frozen)   :   0.000              
      reference    (frozen)   :   1.000  TeV      
"""

I guess this is probably because of the new `Datasets names` on each model? 
How do we deal with this @QRemy @registerrier ? 



Discussion:
I see this can be solved by doing a `Datasets.copy`, ie,  `ds1 = [dataset_fit].copy()[0]`.
But this still seems to be a recipe for disaster. We should then prevent users from accessing `MapDataset.copy()` (and even read and write maybe?) and everything goes through `Datasets`?
The problem probably comes from the change in dataset name that de-sync the background model from the dataset
Could you try
`ds2 = dataset_fit.copy(name=dataset_fit.name)`
or alternatively add 
`ds2.background_model.datasets_names = [ds2.name]` 
Thanks @QRemy ! Both work perfectly.
Maybe it will be good the implement the second solution within gammapy?
Yes I'll do that. Probably have to look for and replace the name in datasets_names for all the models (if not None).
@AtreyeeS I think what causes the unexpected behaviour here is the implicit assumption, that `MapDataset.copy()` copies the model as well, which is not the case.

The question of model handling / maintenance for datasets is unaddressed in general. Currently there is no well defined behaviour on what happens to the model on many dataset operations like:

- `MapDataset.copy()`
- `MapDataset.to_image()`
- `MapDataset.cutout()`
- `MapDataset.to_spectrum_dataset()`
- `MapDataset.stack()`
- etc.

I think we need a uniform strategy on how to handle the model on dataset operations. I think there are a few options:
1. Reset the model to `None`. This is certainly the easiest option, but maybe slightly inconvenient, as the user has to redefine the model, but at least it is explicit and the model maintenance is completely one the use side.
2. Do Nothing to the model. I think this is mostly what we do now (probably not consistently). That's also what causes this "bug" here. I think it can be valid choice and in this case the behaviour of keeping the dataset names would be alright.
3. Try to implement an equivalent operation on the model. This might be the most convenient option to the user, but I'm not sure this can be implemented in all cases, because models are shared between datasets. E.g. what happens if one calls `.to_image()` on one dataset, the model is modified, but it's also part of dataset, which is not modified. This is almost impossible to maintain and probably causes even more confusion.
4. Possibly even remove the model attribute from the dataset and pass it to the methods as needed...

In general this is strongly connected to the question of the global model object, that we couldn't resolve up to now yet. Of course we can fix individual behaviours like here, but we have to resolve the connection between datasets and the model definition in general. I think the main open question is whether the model object is independent of the dataset or "owned" and shared by individual datasets. Some efforts (e.g. the `datasets_names` attribute) went into the direction of making the model object more independent of the dataset object, currently we are still in an in between state.

With  the solution, where the model objects are "owned" by the dataset object, I'm sure we will get into trouble, when we implement parallel evaluation of datasets, where dataset evaluation runs in different Python processes. If a model is changed in one process, it's hard forward the changes to the other processes. From more recent experiences, I'm leaning more and more to a solution where the model object becomes independent of the datasets and is maintained independently. In the most strict implementation this would mean the the model object is removed from the `MapDataset` and passed to the methods like `MapDataset.npred(model)` etc. 

It would be nice @AtreyeeS , @QRemy  and @registerrier if we could collect a few more thoughts and use cases here, that we can work towards a uniform solution more quickly...


For now, I would preserve the models for `.copy()`  because the geom do not change, and reset the models in the other cases. It's somehow already the case for `.to_image()`, `.cutout()` and `.to_spectrum_dataset()`.  where only the background_model is adapted and passed as model. The behavior  of `.stack()` is different, there is no update of the models so it may preserve the models of the base dataset but not those of the other.

We can't do all the models maintenance when the geom change because it's very dependent of each problem and the user should know and control which models are preserved, discarded or adapted on these operations.  Maybe later we could add functionalities to help filtering and adapting the models.
The global question of whether models should be independent of datasets needs some more thought from my side. A situation like the present case, where `dataset.models` lists the background model, but then, does not apply it (because `background_model.datasets_names` points to a different dataset) is something I find very confusing.

If we decide on a case by case approach as proposed by @QRemy , I would tend to prefer:

1. `MapDataset.copy()` keeps all the models, and preferably, does a hard copy of the models. I wanted to use the `copy` option to keep an in memory copy of my dataset at a certain stage after fitting, before I tried something more adventurous with the models.

2. `MapDataset.to_image()` also keeps the models, because otherwise the total `npred` is not accessible. But there is already and open issue on this -  `to_image` will probably need more adaptation as well, see #2865 

3. `MapDataset.cutout()` : As a user, I would naively prefer that the models are preserved, but I think it is okay to discard it here. But then, we should provide some convenience functions/at least documentation of adding back the models

4. `MapDataset.to_spectum_dataset` should not keep the models. I think have spatial models defined on a spectral dataset would tend to be confusing. Or what happens in case of a `SkyDiffuseCube`? 

5. `MapDataset.stack`: A priori, I see no problem in keeping the models in this case. But I do not know if there is a specific use case for it?

6. `MapDataset.write`: For serialisation issues, we have decided that this writes only the maps and not the models to disc. This is indeed reasonable, but I think this adds a high chance of error. I would prefer to keep this option hidden from the general user and encourage only `Datasets.read/write`
For most of the cases, there is no point in keeping the model. 
The main issue is always what to do with `BackgroundModel` which is in-between an IRF and a model. We should probably explicitly separate between the `background` cube (provided by the Makers) and the background model. Then by default, the `BackgroundModel` of a `Dataset` uses its own  `background` cube. 

Resolving patch:
diff --git a/gammapy/datasets/core.py b/gammapy/datasets/core.py
--- a/gammapy/datasets/core.py
+++ b/gammapy/datasets/core.py
@@ -64,7 +64,17 @@ def stat_array(self):
     def copy(self, name=None):
         """A deep copy."""
         new = copy.deepcopy(self)
-        new._name = make_name(name)
+        name = make_name(name)
+        new._name = name
+        # propagate new dataset name
+        if new.models is not None:
+            for m in new.models:
+                if m.datasets_names is not None:
+                    for k, d in enumerate(m.datasets_names):
+                        if d == self.name:
+                            m.datasets_names[k] = name
+                    if hasattr(new, "background_model") and m == new.background_model:
+                        m._name = name + "-bkg"
         return new
 
     @staticmethod
diff --git a/gammapy/datasets/map.py b/gammapy/datasets/map.py
--- a/gammapy/datasets/map.py
+++ b/gammapy/datasets/map.py
@@ -417,6 +417,9 @@ def stack(self, other):
         if self.background_model and background_model:
             self._background_model.map *= self.mask_safe
             self._background_model.stack(background_model, other.mask_safe)
+            self.models = Models([self.background_model])
+        else:
+            self.models = None
 
         if self.mask_safe is not None and other.mask_safe is not None:
             self.mask_safe.stack(other.mask_safe)


Test patch:
diff --git a/gammapy/datasets/tests/test_io.py b/gammapy/datasets/tests/test_io.py
--- a/gammapy/datasets/tests/test_io.py
+++ b/gammapy/datasets/tests/test_io.py
@@ -71,3 +71,6 @@ def test_datasets_to_io(tmp_path):
         datasets.models["background_irf_g09"].covariance,
         datasets.models["background_irf_gc"].covariance,
     )
+
+    dataset_copy = dataset0.copy(name="dataset0-copy")
+    assert dataset_copy.background_model.datasets_names == ["dataset0-copy"]
diff --git a/gammapy/datasets/tests/test_map.py b/gammapy/datasets/tests/test_map.py
--- a/gammapy/datasets/tests/test_map.py
+++ b/gammapy/datasets/tests/test_map.py
@@ -512,6 +512,7 @@ def test_stack(geom, geom_etrue):
     assert_allclose(dataset1.background_model.map.data.sum(), 5987)
     assert_allclose(dataset1.exposure.data, 2.0 * dataset2.exposure.data)
     assert_allclose(dataset1.mask_safe.data.sum(), 20000)
+    assert len(dataset1.models) == 1
 
 
 def to_cube(image):
@@ -823,10 +824,10 @@ def test_names(geom, geom_etrue, sky_model):
     assert dataset2.background_model
     dataset2 = dataset1.copy(name="dataset2")
     assert dataset2.name == "dataset2"
-    assert dataset2.background_model.name == "bkg1"
-    assert dataset1.background_model is not dataset2.background_model
-    assert dataset1.models.names == dataset2.models.names
-    assert dataset1.models is not dataset2.models
+    assert dataset2.background_model.name == "dataset2-bkg"
+    assert dataset2.background_model is not dataset1.background_model
+    assert dataset2.models.names == ["model1", "model2", "dataset2-bkg"]
+    assert dataset2.models is not dataset1.models
 
 
 def test_stack_dataset_dataset_on_off():