Repository: cta-observatory/ctapipe
Pull request #1973 (2022-06-30 12:36:41)

ObsConfig data model
## Use case:

Access the observation information for both simulations and real data in a common way, from an `EventSource` or other io component.  

## Description

This PR will create an observation configuration data model that can be used by real or simulated observations.

This may not  end up being an exact 1-1 mapping of the CTA SchedulingBlock/ObservationBlock model as there are some issues with the current version, and we would also like to support non-CTA instruments.

Two new Components are defined: `SchedulingBlockContainer` and `ObservationBlockContainer`
```
ctapipe.containers.SchedulingBlockContainer:
                         sb_id: Scheduling block ID with default
                                18446744073709551615 with type <class
                                'numpy.uint64'>
                       sb_type: Type of scheduling block with default
                                SchedulingBlockType.UNKNOWN with type <enum
                                'SchedulingBlockType'>
                   producer_id: Origin of the sb_id, i.e. name of the telescope
                                site or 'simulation' with default unknown with
                                type <class 'str'>
                observing_mode: Defines how observations within the Scheduling
                                Block are distributed in space with default
                                ObservingMode.UNKNOWN with type <enum
                                'ObservingMode'>
                 pointing_mode: Defines how the telescope drives move with
                                default PointingMode.UNKNOWN with type <enum
                                'PointingMode'>

ctapipe.containers.ObservationBlockContainer:
                        obs_id: Observation Block ID with default
                                18446744073709551615 with type <class
                                'numpy.uint64'>
                         sb_id: ID of the parent SchedulingBlock with default
                                18446744073709551615 with type <class
                                'numpy.uint64'>
                   producer_id: Origin of the obs_id, i.e. name of the telescope
                                site or 'simulation' with default unknown with
                                type <class 'str'>
                         state: State of this OB with default
                                ObservationBlockState.UNKNOWN with type <enum
                                'ObservationBlockState'>
         subarray_pointing_lat: latitude of the nominal center coordinate of
                                this observation with default nan deg [deg]
         subarray_pointing_lon: longitude of the nominal center coordinate of
                                this observation with default nan deg [deg]
       subarray_pointing_frame: Frame in which the subarray_target is non-
                                moving. If the frame is altaz, The meaning of
                                lat,lon is altitude,azimuth.  If it is icrs, the
                                meaning is RA,Dec with default
                                CoordinateFrameType.UNKNOWN with type <enum
                                'CoordinateFrameType'>
            scheduled_duration: expected duration from scheduler with default
                                nan min [min]
          scheduled_start_time: expected start time from scheduler with default
                                0.0
             actual_start_time: true start time with default 0.0
               actual_duration: true duration with default nan min [min]
```


The tables of observation and scheduling blocks are also written do the `/configuration/observation` group of all output files

## Example

```py3
from pprint import pprint
from ctapipe.io import EventSource

with EventSource("gamma.simtel.gz") as source:
    print([x.as_dict() for x in source.scheduling_blocks.values()])
    print([x.as_dict() for x in source.observation_blocks.values()])
```
```
[{'observing_mode': <ObservingMode.UNKNOWN: -1>,
  'pointing_mode': <PointingMode.DRIFT: 1>,
  'producer_id': 'simulation',
  'sb_id': 5,
  'sb_type': <SchedulingBlockType.OBSERVATION: 0>}]

[{'actual_duration': <Quantity nan min>,
  'actual_start_time': <Time object: scale='tai' format='mjd' value=0.0>,
  'obs_id': 5,
  'producer_id': 'simulation',
  'sb_id': 5,
  'scheduled_duration': <Quantity nan min>,
  'scheduled_start_time': <Time object: scale='tai' format='mjd' value=0.0>,
  'state': <ObservationBlockState.COMPLETED_SUCCEDED: 1>,
  'subarray_pointing_frame': <CoordinateFrameType.ALTAZ: 0>,
  'subarray_pointing_lat': <Quantity 1.2217305 deg>,
  'subarray_pointing_lon': <Quantity 0. deg>}]
```

# Todo:

- [x]  Have `EventSources` fill this information for each observation they provide
- [x] Ensure that `ctapipe-merge`  can merge this info
- [x] Decide how to store the info for DIVERGENT/CONVERGENT modes, at minimum using a convergence point and depth (maybe we don't need special modes for this, and just define convergence_depth=0 == PARALLEL)

This can later be used in #1918 for SimTelEventSource to provide the pointing info

review-notebook-app[bot]: Check out this pull request on&nbsp; <a href="https://app.reviewnb.com/cta-observatory/ctapipe/pull/1973"><img align="absmiddle"  alt="ReviewNB" height="28" class="BotMessageButtonImage" src="https://raw.githubusercontent.com/ReviewNB/support/master/images/button_reviewnb.png"/></a> 

 See visual diffs & provide feedback on Jupyter Notebooks. 

---

 <i>Powered by <a href='https://www.reviewnb.com/?utm_source=gh'>ReviewNB</a></i>
kosack: Another question: 

`SimTelEventSource` and `HDF5EventSource` only return one` SimulationConfigurationContainer` via their `simulation_config` property .   the `observation_config` should be essentially the same interface.   Should all three support returning a tuple of  both observation and simulation configs? I guess the answer is yes, but that is not what happens now. 
codecov[bot]: # [Codecov](https://codecov.io/gh/cta-observatory/ctapipe/pull/1973?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=cta-observatory) Report
> Merging [#1973](https://codecov.io/gh/cta-observatory/ctapipe/pull/1973?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=cta-observatory) (1c3176b) into [master](https://codecov.io/gh/cta-observatory/ctapipe/commit/a9d2a00b242f58db3f328f4d1f3fc34e228dad57?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=cta-observatory) (a9d2a00) will **decrease** coverage by `0.00%`.
> The diff coverage is `92.40%`.

```diff
@@            Coverage Diff             @@
##           master    #1973      +/-   ##
==========================================
- Coverage   92.42%   92.42%   -0.01%     
==========================================
  Files         197      197              
  Lines       16030    16135     +105     
==========================================
+ Hits        14816    14912      +96     
- Misses       1214     1223       +9     
```


| [Impacted Files](https://codecov.io/gh/cta-observatory/ctapipe/pull/1973?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=cta-observatory) | Coverage Δ | |
|---|---|---|
| [ctapipe/io/toymodel.py](https://codecov.io/gh/cta-observatory/ctapipe/pull/1973/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=cta-observatory#diff-Y3RhcGlwZS9pby90b3ltb2RlbC5weQ==) | `90.36% <71.42%> (-3.23%)` | :arrow_down: |
| [ctapipe/io/tests/test\_event\_source.py](https://codecov.io/gh/cta-observatory/ctapipe/pull/1973/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=cta-observatory#diff-Y3RhcGlwZS9pby90ZXN0cy90ZXN0X2V2ZW50X3NvdXJjZS5weQ==) | `95.72% <75.00%> (-0.77%)` | :arrow_down: |
| [ctapipe/tools/tests/test\_process.py](https://codecov.io/gh/cta-observatory/ctapipe/pull/1973/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=cta-observatory#diff-Y3RhcGlwZS90b29scy90ZXN0cy90ZXN0X3Byb2Nlc3MucHk=) | `94.96% <75.00%> (-0.63%)` | :arrow_down: |
| [ctapipe/io/eventsource.py](https://codecov.io/gh/cta-observatory/ctapipe/pull/1973/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=cta-observatory#diff-Y3RhcGlwZS9pby9ldmVudHNvdXJjZS5weQ==) | `89.07% <78.57%> (-1.67%)` | :arrow_down: |
| [ctapipe/io/hdf5eventsource.py](https://codecov.io/gh/cta-observatory/ctapipe/pull/1973/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=cta-observatory#diff-Y3RhcGlwZS9pby9oZGY1ZXZlbnRzb3VyY2UucHk=) | `90.38% <94.44%> (+0.06%)` | :arrow_up: |
| [ctapipe/containers.py](https://codecov.io/gh/cta-observatory/ctapipe/pull/1973/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=cta-observatory#diff-Y3RhcGlwZS9jb250YWluZXJzLnB5) | `100.00% <100.00%> (ø)` | |
| [ctapipe/io/datawriter.py](https://codecov.io/gh/cta-observatory/ctapipe/pull/1973/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=cta-observatory#diff-Y3RhcGlwZS9pby9kYXRhd3JpdGVyLnB5) | `90.19% <100.00%> (+0.22%)` | :arrow_up: |
| [ctapipe/io/simteleventsource.py](https://codecov.io/gh/cta-observatory/ctapipe/pull/1973/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=cta-observatory#diff-Y3RhcGlwZS9pby9zaW10ZWxldmVudHNvdXJjZS5weQ==) | `96.74% <100.00%> (+0.12%)` | :arrow_up: |
| [ctapipe/io/tests/test\_datawriter.py](https://codecov.io/gh/cta-observatory/ctapipe/pull/1973/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=cta-observatory#diff-Y3RhcGlwZS9pby90ZXN0cy90ZXN0X2RhdGF3cml0ZXIucHk=) | `100.00% <100.00%> (ø)` | |
| [ctapipe/io/tests/test\_hdf5eventsource.py](https://codecov.io/gh/cta-observatory/ctapipe/pull/1973/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=cta-observatory#diff-Y3RhcGlwZS9pby90ZXN0cy90ZXN0X2hkZjVldmVudHNvdXJjZS5weQ==) | `97.58% <100.00%> (ø)` | |
| ... and [4 more](https://codecov.io/gh/cta-observatory/ctapipe/pull/1973/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=cta-observatory) | |

------

[Continue to review full report at Codecov](https://codecov.io/gh/cta-observatory/ctapipe/pull/1973?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=cta-observatory).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=cta-observatory)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/cta-observatory/ctapipe/pull/1973?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=cta-observatory). Last update [ba821d2...1c3176b](https://codecov.io/gh/cta-observatory/ctapipe/pull/1973?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=cta-observatory). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=cta-observatory).


Patch:
diff --git a/ctapipe/containers.py b/ctapipe/containers.py
index 774d6e65..3f4431dc 100644
--- a/ctapipe/containers.py
+++ b/ctapipe/containers.py
@@ -57,20 +57,85 @@ __all__ = [
     "StatisticsContainer",
     "IntensityStatisticsContainer",
     "PeakTimeStatisticsContainer",
+    "SchedulingBlockContainer",
+    "ObservationBlockContainer",
+    "ObservingMode",
+    "ObservationBlockState",
 ]
 
 
 # see https://github.com/astropy/astropy/issues/6509
 NAN_TIME = Time(0, format="mjd", scale="tai")
 
+#: Used for unsigned integer obs_id or sb_id default values:
+UNKNOWN_ID = np.uint64(np.iinfo(np.uint64).max)
 
-class EventType(enum.Enum):
-    """Enum of EventTypes as defined in the CTA Data Model
 
-    These numbers come from  the document *CTA R1/Event Data Model Specification*
-    version 1 revision C.  They may be updated in future revisions
+class SchedulingBlockType(enum.Enum):
+    """
+    Types of Scheduling Block
+    """
+
+    UNKNOWN = -1
+    OBSERVATION = 0
+    CALIBRATION = 1
+    ENGINEERING = 2
+
+
+class ObservationBlockState(enum.Enum):
+    """Observation Block States. Part of the Observation Configuration data
+    model.
+    """
+
+    UNKNOWN = -1
+    FAILED = 0
+    COMPLETED_SUCCEDED = 1
+    COMPLETED_CANCELED = 2
+    COMPLETED_TRUNCATED = 3
+    ARCHIVED = 4
+
+
+class ObservingMode(enum.Enum):
+    """How a scheduling block is observed. Part of the Observation Configuration
+    data model.
+
+    """
+
+    UNKNOWN = -1
+    WOBBLE = 0
+    ON_OFF = 1
+    GRID = 2
+    CUSTOM = 3
+
+
+class PointingMode(enum.Enum):
+    """Describes how the telescopes move. Part of the Observation Configuration
+    data model.
+
+    """
+
+    UNKNOWN = -1
+    #: drives track a point that moves with the sky
+    TRACK = 0
+    #: drives stay fixed at an alt/az point while the sky drifts by
+    DRIFT = 1
+
+
+class CoordinateFrameType(enum.Enum):
+    """types of coordinate frames used in ObservationBlockContainers. Part of
+    the Observation Configuration data model.
+
     """
 
+    UNKNOWN = -1
+    ALTAZ = 0
+    ICRS = 1
+    GALACTIC = 2
+
+
+class EventType(enum.Enum):
+    """Enum of EventTypes as defined in the CTA Data Model [cta_r1event]_"""
+
     # calibrations are 0-15
     FLATFIELD = 0
     SINGLE_PE = 1
@@ -79,14 +144,14 @@ class EventType(enum.Enum):
     ELECTRONIC_PEDESTAL = 4
     OTHER_CALIBRATION = 15
 
-    # For mono-telescope triggers (not used in MC)
+    #: For mono-telescope triggers (not used in MC)
     MUON = 16
     HARDWARE_STEREO = 17
 
-    # ACADA (DAQ) software trigger
+    #: ACADA (DAQ) software trigger
     DAQ = 24
 
-    # Standard Physics  stereo trigger
+    #: Standard Physics  stereo trigger
     SUBARRAY = 32
 
     UNKNOWN = 255
@@ -1143,3 +1208,78 @@ class ArrayEventContainer(Container):
         default_factory=MonitoringContainer,
         description="container for event-wise monitoring data (MON)",
     )
+
+
+class SchedulingBlockContainer(Container):
+    """Stores information about the scheduling block. This is a simplified
+    version of the SB model, only storing what is necessary for analysis. From
+    [cta_sb_ob]_
+
+    """
+
+    default_prefix = ""
+    sb_id = Field(UNKNOWN_ID, "Scheduling block ID", type=np.uint64)
+    sb_type = Field(
+        SchedulingBlockType.UNKNOWN,
+        description="Type of scheduling block",
+        type=SchedulingBlockType,
+    )
+    producer_id = Field(
+        "unknown",
+        "Origin of the sb_id, i.e. name of the telescope site or 'simulation'",
+        type=str,
+    )
+    observing_mode = Field(
+        ObservingMode.UNKNOWN,
+        "Defines how observations within the Scheduling Block are distributed in space",
+        type=ObservingMode,
+    )
+    pointing_mode = Field(
+        PointingMode.UNKNOWN, "Defines how the telescope drives move", type=PointingMode
+    )
+
+
+class ObservationBlockContainer(Container):
+    """Stores information about the observation"""
+
+    default_prefix = ""
+    obs_id = Field(UNKNOWN_ID, "Observation Block ID", type=np.uint64)
+    sb_id = Field(UNKNOWN_ID, "ID of the parent SchedulingBlock", type=np.uint64)
+    producer_id = Field(
+        "unknown",
+        "Origin of the obs_id, i.e. name of the telescope site or 'simulation'",
+        type=str,
+    )
+
+    state = Field(
+        ObservationBlockState.UNKNOWN, "State of this OB", type=ObservationBlockState
+    )
+
+    subarray_pointing_lat = Field(
+        nan * u.deg,
+        "latitude of the nominal center coordinate of this observation",
+        unit=u.deg,
+    )
+
+    subarray_pointing_lon = Field(
+        nan * u.deg,
+        "longitude of the nominal center coordinate of this observation",
+        unit=u.deg,
+    )
+
+    subarray_pointing_frame = Field(
+        CoordinateFrameType.UNKNOWN,
+        (
+            "Frame in which the subarray_target is non-moving. If the frame is ALTAZ, "
+            "the meaning of (lon,lat) is (azimuth, altitude) while for ICRS it is "
+            "(right-ascension, declination)"
+        ),
+        type=CoordinateFrameType,
+    )
+
+    scheduled_duration = Field(
+        nan * u.min, "expected duration from scheduler", unit=u.min
+    )
+    scheduled_start_time = Field(NAN_TIME, "expected start time from scheduler")
+    actual_start_time = Field(NAN_TIME, "true start time")
+    actual_duration = Field(nan * u.min, "true duration", unit=u.min)
diff --git a/ctapipe/io/datawriter.py b/ctapipe/io/datawriter.py
index 55735465..0f84f44e 100644
--- a/ctapipe/io/datawriter.py
+++ b/ctapipe/io/datawriter.py
@@ -40,7 +40,7 @@ def _get_tel_index(event, tel_id):
     )
 
 
-# define the version of the DL1 data model written here. This should be updated
+# define the version of the data model written here. This should be updated
 # when necessary:
 # - increase the major number if there is a breaking change to the model
 #   (meaning readers need to update scripts)
@@ -61,6 +61,7 @@ DATA_MODEL_CHANGE_HISTORY = """
           - The reference_location (EarthLocation origin of the telescope coordinates)
             is now included in SubarrayDescription
           - Only unique optics are stored in the optics table
+          - include observation configuration
 - v3.0.0: reconstructed core uncertainties splitted in their X-Y components
 - v2.2.0: added R0 and R1 outputs
 - v2.1.0: hillas and timing parameters are per default saved in telescope frame (degree) as opposed to camera frame (m)
@@ -137,7 +138,7 @@ def write_reference_metadata_headers(
 
 class DataWriter(Component):
     """
-    Serialize a sequence of events into a HDF5 DL1 file, in the correct format
+    Serialize a sequence of events into a HDF5 file, in the correct format
 
     Examples
     --------
@@ -268,6 +269,7 @@ class DataWriter(Component):
 
     def _setup_outputfile(self):
         self._subarray.to_hdf(self._writer.h5file)
+        self._write_scheduling_and_observation_blocks()
         if self._is_simulation:
             self._write_simulation_configuration()
 
@@ -321,7 +323,7 @@ class DataWriter(Component):
 
     def finish(self):
         """called after all events are done"""
-        self.log.info("Finishing DL1 output")
+        self.log.info("Finishing output")
         if not self._at_least_one_event:
             self.log.warning("No events have been written to the output file")
         if self._writer:
@@ -496,6 +498,21 @@ class DataWriter(Component):
             writer.write("dl1/monitoring/subarray/pointing", [event.trigger, pnt])
             self._last_pointing = current_pointing
 
+    def _write_scheduling_and_observation_blocks(self):
+        """write out SB and OB info"""
+
+        self.log.debug(
+            "writing %d sbs and %d obs",
+            len(self.event_source.scheduling_blocks.values()),
+            len(self.event_source.observation_blocks.values()),
+        )
+
+        for sb in self.event_source.scheduling_blocks.values():
+            self._writer.write("configuration/observation/scheduling_block", sb)
+
+        for ob in self.event_source.observation_blocks.values():
+            self._writer.write("configuration/observation/observation_block", ob)
+
     def _write_simulation_configuration(self):
         """
         Write the simulation headers to a single row of a table. Later
diff --git a/ctapipe/io/eventsource.py b/ctapipe/io/eventsource.py
index 6eb6ed88..7115e8b7 100644
--- a/ctapipe/io/eventsource.py
+++ b/ctapipe/io/eventsource.py
@@ -3,11 +3,16 @@ Handles reading of different event/waveform containing files
 """
 import warnings
 from abc import abstractmethod
-from typing import Generator, List, Tuple
+from typing import Dict, Generator, List, Tuple
 
 from traitlets.config.loader import LazyConfigValue
 
-from ..containers import ArrayEventContainer
+from ..containers import (
+    ArrayEventContainer,
+    ObservationBlockContainer,
+    SchedulingBlockContainer,
+    SimulationConfigContainer,
+)
 from ..core import Provenance, ToolConfigurationError
 from ..core.component import Component, find_config_in_hierarchy, non_abstract_children
 from ..core.traits import CInt, Int, Path, Set, TraitError, Undefined
@@ -205,11 +210,38 @@ class EventSource(Component):
 
         """
 
+    @property
+    def simulation_config(self) -> Dict[int, SimulationConfigContainer]:
+        """The simulation configurations of all observations provided by the
+        EventSource, or None if the source does not provide simulated data
+
+        Returns
+        -------
+        Dict[int,ctapipe.containers.SimulationConfigContainer] | None
+        """
+        return None
+
+    @property
+    @abstractmethod
+    def observation_blocks(self) -> Dict[int, ObservationBlockContainer]:
+        """
+        Obtain the ObservationConfigurations from the EventSource, indexed by obs_id
+        """
+        pass
+
+    @property
+    @abstractmethod
+    def scheduling_blocks(self) -> Dict[int, SchedulingBlockContainer]:
+        """
+        Obtain the ObservationConfigurations from the EventSource, indexed by obs_id
+        """
+        pass
+
     @property
     @abstractmethod
     def is_simulation(self) -> bool:
         """
-        Weither the currently opened file is simulated
+        Whether the currently opened file is simulated
 
         Returns
         -------
@@ -240,7 +272,6 @@ class EventSource(Component):
         return any(dl in self.datalevels for dl in datalevels)
 
     @property
-    @abstractmethod
     def obs_ids(self) -> List[int]:
         """
         The observation ids of the runs located in the file
@@ -250,6 +281,7 @@ class EventSource(Component):
         -------
         list[int]
         """
+        return list(self.observation_blocks.keys())
 
     @abstractmethod
     def _generator(self) -> Generator[ArrayEventContainer, None, None]:
diff --git a/ctapipe/io/hdf5eventsource.py b/ctapipe/io/hdf5eventsource.py
index 423c9db4..fafaeece 100644
--- a/ctapipe/io/hdf5eventsource.py
+++ b/ctapipe/io/hdf5eventsource.py
@@ -1,6 +1,7 @@
 import logging
-from pathlib import Path
 from functools import lru_cache
+from pathlib import Path
+from typing import Dict
 
 import astropy.units as u
 import numpy as np
@@ -21,11 +22,13 @@ from ..containers import (
     IntensityStatisticsContainer,
     LeakageContainer,
     MorphologyContainer,
+    ObservationBlockContainer,
     ParticleClassificationContainer,
     PeakTimeStatisticsContainer,
     R1CameraContainer,
     ReconstructedEnergyContainer,
     ReconstructedGeometryContainer,
+    SchedulingBlockContainer,
     SimulatedEventContainer,
     SimulatedShowerContainer,
     SimulationConfigContainer,
@@ -163,6 +166,10 @@ class HDF5EventSource(EventSource):
         else:
             self._subarray = self._full_subarray
         self._simulation_configs = self._parse_simulation_configs()
+        (
+            self._scheduling_block,
+            self._observation_block,
+        ) = self._parse_sb_and_ob_configs()
         self.datamodel_version = self.file_.root._v_attrs[
             "CTA PRODUCT DATA MODEL VERSION"
         ]
@@ -243,7 +250,15 @@ class HDF5EventSource(EventSource):
         return list(np.unique(self.file_.root.dl1.event.subarray.trigger.col("obs_id")))
 
     @property
-    def simulation_config(self):
+    def scheduling_blocks(self) -> Dict[int, SchedulingBlockContainer]:
+        return self._scheduling_block
+
+    @property
+    def observation_blocks(self) -> Dict[int, ObservationBlockContainer]:
+        return self._observation_block
+
+    @property
+    def simulation_config(self) -> Dict[int, SimulationConfigContainer]:
         """
         Returns the simulation config(s) as
         a dict mapping obs_id to the respective config.
@@ -272,16 +287,32 @@ class HDF5EventSource(EventSource):
             default_prefix = ""
             obs_id = Field(-1)
 
-        simulation_configs = {}
         if "simulation" in self.file_.root.configuration:
             reader = HDF5TableReader(self.file_).read(
                 "/configuration/simulation/run",
                 containers=(SimulationConfigContainer, ObsIdContainer),
             )
-            for (config, index) in reader:
-                simulation_configs[index.obs_id] = config
+            return {index.obs_id: config for (config, index) in reader}
+        else:
+            return {}
+
+    def _parse_sb_and_ob_configs(self):
+        """read Observation and Scheduling block configurations"""
+
+        sb_reader = HDF5TableReader(self.file_).read(
+            "/configuration/observation/scheduling_block",
+            containers=SchedulingBlockContainer,
+        )
+
+        scheduling_blocks = {sb.sb_id: sb for sb in sb_reader}
+
+        ob_reader = HDF5TableReader(self.file_).read(
+            "/configuration/observation/observation_block",
+            containers=ObservationBlockContainer,
+        )
+        observation_blocks = {ob.obs_id: ob for ob in ob_reader}
 
-        return simulation_configs
+        return scheduling_blocks, observation_blocks
 
     def _generator(self):
         """
diff --git a/ctapipe/io/simteleventsource.py b/ctapipe/io/simteleventsource.py
index b7196b4b..2b4b141e 100644
--- a/ctapipe/io/simteleventsource.py
+++ b/ctapipe/io/simteleventsource.py
@@ -3,6 +3,7 @@ import warnings
 from gzip import GzipFile
 from io import BufferedReader
 from pathlib import Path
+from typing import Dict
 
 import numpy as np
 from astropy import units as u
@@ -14,12 +15,19 @@ from eventio.simtel.simtelfile import SimTelFile
 from ..calib.camera.gainselection import GainSelector
 from ..containers import (
     ArrayEventContainer,
+    CoordinateFrameType,
     EventIndexContainer,
     EventType,
+    ObservationBlockContainer,
+    ObservationBlockState,
+    ObservingMode,
     PixelStatusContainer,
     PointingContainer,
+    PointingMode,
     R0CameraContainer,
     R1CameraContainer,
+    SchedulingBlockContainer,
+    SchedulingBlockType,
     SimulatedCameraContainer,
     SimulatedEventContainer,
     SimulatedShowerContainer,
@@ -351,6 +359,10 @@ class SimTelEventSource(EventSource):
         self._subarray_info = self.prepare_subarray_info(
             self.file_.telescope_descriptions, self.file_.header
         )
+        (
+            self._scheduling_blocks,
+            self._observation_blocks,
+        ) = self._fill_scheduling_and_observation_blocks()
         self._simulation_config = self._parse_simulation_header()
         self.start_pos = self.file_.tell()
 
@@ -374,13 +386,22 @@ class SimTelEventSource(EventSource):
         return (DataLevel.R0, DataLevel.R1)
 
     @property
-    def obs_ids(self):
-        # ToDo: This does not support merged simtel files!
-        return [self.file_.header["run"]]
+    def simulation_config(self) -> Dict[int, SimulationConfigContainer]:
+        return self._simulation_config
 
     @property
-    def simulation_config(self) -> SimulationConfigContainer:
-        return self._simulation_config
+    def observation_blocks(self) -> Dict[int, ObservationBlockContainer]:
+        """
+        Obtain the ObservationConfigurations from the EventSource, indexed by obs_id
+        """
+        return self._observation_blocks
+
+    @property
+    def scheduling_blocks(self) -> Dict[int, SchedulingBlockContainer]:
+        """
+        Obtain the ObservationConfigurations from the EventSource, indexed by obs_id
+        """
+        return self._scheduling_blocks
 
     @property
     def is_stream(self):
@@ -816,6 +837,40 @@ class SimTelEventSource(EventSource):
         )
         return {obs_id: simulation_config}
 
+    def _fill_scheduling_and_observation_blocks(self):
+        """fill scheduling and observation blocks must be run after the
+        simulation config is filled
+        """
+
+        az, alt = self.file_.header["direction"]
+        obs_id = self.file_.header["run"]
+
+        sb_dict = {
+            obs_id: SchedulingBlockContainer(
+                sb_id=np.uint64(obs_id),  # simulations have no SBs, use OB id
+                sb_type=SchedulingBlockType.OBSERVATION,
+                producer_id="simulation",
+                observing_mode=ObservingMode.UNKNOWN,
+                pointing_mode=PointingMode.DRIFT,
+            )
+        }
+
+        ob_dict = {
+            obs_id: ObservationBlockContainer(
+                obs_id=np.uint64(obs_id),
+                sb_id=np.uint64(obs_id),  # see comment above
+                producer_id="simulation",
+                state=ObservationBlockState.COMPLETED_SUCCEDED,
+                subarray_pointing_lat=alt * u.rad,
+                subarray_pointing_lon=az * u.rad,
+                subarray_pointing_frame=CoordinateFrameType.ALTAZ,
+                actual_start_time=Time(self.file_.header["time"], format="unix"),
+                scheduled_start_time=Time(self.file_.header["time"], format="unix"),
+            )
+        }
+
+        return sb_dict, ob_dict
+
     @staticmethod
     def _fill_simulated_event_information(array_event):
         mc_event = array_event["mc_event"]
diff --git a/ctapipe/io/toymodel.py b/ctapipe/io/toymodel.py
index 935f6560..552fd3de 100644
--- a/ctapipe/io/toymodel.py
+++ b/ctapipe/io/toymodel.py
@@ -3,16 +3,22 @@
 Create a toymodel event stream of array events
 """
 import logging
+from typing import Dict
 
-import numpy as np
 import astropy.units as u
+import numpy as np
 
-from ..containers import ArrayEventContainer, DL1CameraContainer, EventIndexContainer
-from ..core import traits
-from ..core import TelescopeComponent
+from ..containers import (
+    ArrayEventContainer,
+    DL1CameraContainer,
+    EventIndexContainer,
+    ObservationBlockContainer,
+    SchedulingBlockContainer,
+)
+from ..core import TelescopeComponent, traits
 from ..image import toymodel
-from .eventsource import EventSource
 from .datalevels import DataLevel
+from .eventsource import EventSource
 
 logger = logging.getLogger(__name__)
 
@@ -58,16 +64,12 @@ class ToyEventSource(EventSource, TelescopeComponent):
 
     @staticmethod
     def calc_width(eccentricity, length):
-        return length * np.sqrt(1 - eccentricity ** 2)
+        return length * np.sqrt(1 - eccentricity**2)
 
     @property
     def subarray(self):
         return self._subarray
 
-    @property
-    def obs_ids(self):
-        return [-1]
-
     @property
     def is_simulation(self):
         return True
@@ -76,6 +78,16 @@ class ToyEventSource(EventSource, TelescopeComponent):
     def datalevels(self):
         return (DataLevel.DL1_IMAGES,)
 
+    @property
+    def scheduling_blocks(self) -> Dict[int, SchedulingBlockContainer]:
+        sb = SchedulingBlockContainer(producer_id="ctapipe toymodel")
+        return {sb.sb_id: sb}
+
+    @property
+    def observation_blocks(self) -> Dict[int, ObservationBlockContainer]:
+        ob = ObservationBlockContainer(producer_id="ctapipe toymodel")
+        return {ob.ob_id: ob}
+
     @subarray.setter
     def subarray(self, value):
         self._subarray = value
@@ -130,7 +142,7 @@ class ToyEventSource(EventSource, TelescopeComponent):
 
             psi = self.rng.uniform(0, 360)
             shower_area_ratio = (
-                2 * np.pi * width * length / cam.pix_area.mean().to_value(u.m ** 2)
+                2 * np.pi * width * length / cam.pix_area.mean().to_value(u.m**2)
             )
             intensity = self.rng.poisson(50) * shower_area_ratio
             skewness = self.rng.uniform(
diff --git a/ctapipe/tools/merge.py b/ctapipe/tools/merge.py
index cc00f7c9..0b7ca3f8 100644
--- a/ctapipe/tools/merge.py
+++ b/ctapipe/tools/merge.py
@@ -39,6 +39,11 @@ optional_nodes = {
     "/dl2/event/subarray/geometry",
 }
 
+observation_configuration_nodes = {
+    "/configuration/observation/observation_block",
+    "/configuration/observation/scheduling_block",
+}
+
 simulation_nodes = {
     "/simulation/event/subarray/shower",
     "/simulation/event/telescope/parameters",
@@ -76,6 +81,7 @@ all_nodes = (
     | parameter_nodes
     | simulation_images
     | dl2_subarray_nodes
+    | observation_configuration_nodes
 )
 
 
diff --git a/docs/bibliography.rst b/docs/bibliography.rst
index c7de6ab3..3ae82af0 100644
--- a/docs/bibliography.rst
+++ b/docs/bibliography.rst
@@ -32,3 +32,7 @@ References
     Data Mining and Knowledge Discovery, 12, 29-45, DOI: 10.1007/s10618-005-0024-4
 
 .. [hillas_uncertainties] W. Wittek, "Image Parameters", MAGIC-TDAS internal note 02-03, (2002). 
+
+.. [cta_sb_ob] Oya et al, Scheduling Block Data Model Specification, CTA-SPE-COM-000000-0003, Issue 1, Rev. c
+
+.. [cta_r1event] CTA R1/Event Data Model Specification, version 1 revision C (2022).


Test patch:
diff --git a/ctapipe/io/tests/test_event_source.py b/ctapipe/io/tests/test_event_source.py
index 9a839ff8..7b6df272 100644
--- a/ctapipe/io/tests/test_event_source.py
+++ b/ctapipe/io/tests/test_event_source.py
@@ -1,10 +1,10 @@
 import pytest
-from ctapipe.utils import get_dataset_path
-from ctapipe.io import EventSource, SimTelEventSource, DataLevel
-from traitlets.config.loader import Config
 from traitlets import TraitError
+from traitlets.config.loader import Config
 
 from ctapipe.core import Component
+from ctapipe.io import DataLevel, EventSource, SimTelEventSource
+from ctapipe.utils import get_dataset_path
 
 prod5_path = "gamma_20deg_0deg_run2___cta-prod5-paranal_desert-2147m-Paranal-dark_cone10-100evts.simtel.zst"
 
@@ -36,8 +36,12 @@ class DummyReader(EventSource):
         return False
 
     @property
-    def obs_ids(self):
-        return [1]
+    def scheduling_blocks(self):
+        return dict()
+
+    @property
+    def observation_blocks(self):
+        return dict()
 
     @property
     def datalevels(self):
diff --git a/ctapipe/tools/tests/test_merge.py b/ctapipe/tools/tests/test_merge.py
index 3be885ba..05a18e06 100644
--- a/ctapipe/tools/tests/test_merge.py
+++ b/ctapipe/tools/tests/test_merge.py
@@ -175,3 +175,10 @@ def test_dl2(tmp_path, dl2_shower_geometry_file, dl2_proton_geometry_file):
 
     for col in ("counts", "cumulative_counts"):
         assert np.all(merged_stats[col] == (stats1[col] + stats2[col]))
+
+    # test reading configurations as well:
+    obs = read_table(output, "/configuration/observation/observation_block")
+    sbs = read_table(output, "/configuration/observation/scheduling_block")
+
+    assert len(obs) == 2, "should have two OB entries"
+    assert len(sbs) == 2, "should have two SB entries"
diff --git a/ctapipe/tools/tests/test_process.py b/ctapipe/tools/tests/test_process.py
index 422b28cd..5744cb5a 100644
--- a/ctapipe/tools/tests/test_process.py
+++ b/ctapipe/tools/tests/test_process.py
@@ -169,8 +169,12 @@ def test_stage1_datalevels(tmp_path):
             return True
 
         @property
-        def obs_ids(self):
-            return [1]
+        def scheduling_blocks(self):
+            return dict()
+
+        @property
+        def observation_blocks(self):
+            return dict()
 
         @property
         def subarray(self):