Repository: gammapy/gammapy

Issue:
Documentation on modifying events table
Since it is not possible to rewrite `Observation.events`, we should show some documentation on how to modify the events list. 

1. The minimal solution would be to provide a HowTo entry with maybe the following lines of code

```
import numpy as np
from gammapy.data import DataStore, Observation, EventList

datastore = DataStore.from_dir("$GAMMAPY_DATA/hess-dl3-dr1/")
obs = datastore.obs(23523)

# compute the phase separately, using, eg: PINT
phase = np.random.random(len(obs.events.table))

# attach the phase column to the events table
table = obs.events.table
table["PHASE"] = phase
events_new = EventList(table)

# create a new observation
irfs = {}
for _ in obs.available_irfs:
    irfs[_]= getattr(obs,_)
obs_new = = Observation.create(pointing=obs.pointing_radec, 
                  location=obs.observatory_earth_location,
                  obs_id=obs.obs_id,
                  tstart=obs.tstart,
                  tstop=obs.tstop,
                  irfs=irfs)
```

2. We can modify `Observation.create()` or add a `Observation.copy()` to simplify this. One of the problems with the above lines of code is that most of the information contained in `obs.fixed_pointing_info.meta` is lost in `obs_new`

I can implement either. Preference @registerrier @maxnoe @adonath ?



Discussion:
Why is it not possible?

Wouldn't this work?

```
obs.events.table["PHASE"] = phase
obs.write("...)
```

?
> obs.fixed_pointing_info.meta is lost in obs_new

This is strange, since this should come from the info in the events header table. 
@maxnoe 

> Why is it not possible?

This is not possible because of the lazy loading.
If you explicitly try to set `obs.events = events_new` it raises `AttributeError: can't set attribute`.
What you propose silently fails
In general, I think the behaviour of `Observation.create` is not proper as it does not use the `rad_max` irf 

https://github.com/gammapy/gammapy/blob/89f67e3014c3673704ea6eeb8ad3ad0b844a3426/gammapy/data/observations.py#L223

I am not putting a milestone now, but I think this should be fixed before 1.0, no?
> If you explicitly try to set obs.events = events_new it raises AttributeError: can't set attribute.
What you propose silently fails

I know. But my code does not change `events` to be a new instance. It *mutates* the events table in place:

```
from gammapy.data import Observation
import numpy as np

obs = Observation.read('$GAMMAPY_DATA/joint-crab/dl3/magic/run_05029748_DL3.fits')

# attach phase information
obs.events.table['PHASE'] = np.random.uniform(0, 1, len(obs.events.table))

# write out
obs.write('with_phase.fits.gz')

del obs
# read back
obs_read = Observation.read('with_phase.fits.gz')

print(obs_read.events.table[:10])

```
Output:

```
EVENT_ID        TIME            RA       DEC       ENERGY          PHASE        
                 s             deg       deg        TeV                         
-------- ------------------ --------- --------- ----------- --------------------
     123 333780073.20475286 443.60065 21.996529  0.13071066   0.7546803999856649
     298  333780073.8190223 444.36084  20.54499  0.19850528   0.4674345981813153
     346  333780073.9708974 443.93896 21.038033  0.07445403    0.735522444859453
     422  333780074.2117895 442.59735 23.010326  0.08110707 0.012711786872918185
     423 333780074.21213335  443.5718 22.053316    0.508845  0.21095066372948412
     460 333780074.31514966  443.1999 21.678938 0.069036916  0.05026659888208096
     466   333780074.340594 443.20682 22.045116  0.21033241  0.07602716264256326
     652 333780074.95243436 442.41882 22.056078 0.078639954   0.8853568722590389
     862  333780075.6436403 443.53848  21.27392   0.1934433  0.16584034944275539
     877 333780075.70590734  444.7129 21.414494  0.10719763   0.5457866516497857
```

Okay this is a bit strange

<img width="636" alt="Screenshot 2022-07-04 at 17 55 04" src="https://user-images.githubusercontent.com/32677370/177188292-e8811177-5f89-4b88-9f91-b8189130d49f.png">

It seems the behaviour is different between `Observation.read()` and `Datastore.obs()` 
@AtreyeeS `Observation.read` does not use the lazy loading mechanism, it just actually reads the observation into memory.
One point regarding pulsar phase: DL3 files are source agnotic and phase is a property of an assumed source. There could even be the case where you have two pulsars in the same FoV, which will probably be the case for CTA observations of the galactic plane?

So having a single phase column in dl3 event lists does not really match the data model. You'd need a phase column per assumed object. 

Of course, in most current  cases, there is only one. 

But at least CTA will very probably not have phase information prefilled in DL3 files, since the only relevant information from CTA is the timestamp of the events and the rest has to come from external information about the assumed object(s)...

Resolving patch:
diff --git a/gammapy/data/observations.py b/gammapy/data/observations.py
--- a/gammapy/data/observations.py
+++ b/gammapy/data/observations.py
@@ -187,7 +187,7 @@ def create(
             Stop time of observation as `~astropy.time.Time` or duration
             relative to `reference_time`
         irfs: dict
-            IRFs used for simulating the observation: `bkg`, `aeff`, `psf`, `edisp`
+            IRFs used for simulating the observation: `bkg`, `aeff`, `psf`, `edisp`, `rad_max`
         deadtime_fraction : float, optional
             Deadtime fraction, defaults to 0
         reference_time : `~astropy.time.Time`
@@ -222,6 +222,7 @@ def create(
             bkg=irfs.get("bkg"),
             edisp=irfs.get("edisp"),
             psf=irfs.get("psf"),
+            rad_max=irfs.get("rad_max"),
         )
 
     @property


Test patch:
diff --git a/gammapy/data/tests/test_observations.py b/gammapy/data/tests/test_observations.py
--- a/gammapy/data/tests/test_observations.py
+++ b/gammapy/data/tests/test_observations.py
@@ -244,6 +244,30 @@ def test_observation_cta_1dc():
     assert_allclose(obs.muoneff, 1)
 
 
+@requires_data()
+def test_observation_create_radmax():
+    pointing = SkyCoord(0, 0, unit="deg", frame="galactic")
+    obs = Observation.read("$GAMMAPY_DATA/joint-crab/dl3/magic/run_05029748_DL3.fits")
+    livetime = 5.0 * u.hr
+    deadtime = 0.5
+
+    irfs = {}
+    for _ in obs.available_irfs:
+        irfs[_] = getattr(obs, _)
+
+    obs1 = Observation.create(
+        pointing,
+        irfs=irfs,
+        deadtime_fraction=deadtime,
+        livetime=livetime,
+    )
+
+    assert_skycoord_allclose(obs1.pointing_radec, pointing.icrs)
+    assert_allclose(obs1.observation_live_time_duration, 0.5 * livetime)
+    assert obs1.rad_max is not None
+    assert obs1.psf is None
+
+
 @requires_data()
 def test_observation_read():
     """read event list and irf components from different DL3 files"""