Repository: cta-observatory/ctapipe
Pull request #1949 (2022-06-24 13:44:24)

Rework container prefix handling
This PR changes our prefix handling to make it consistent over the different IO methods. 

Instead of attaching the algorithm name in the TableLoader to avoid duplicated column names, the container prefixes are already set by the algorithms themselves.

This means the prefixed column names are written into the  file and the loader can just read them.

Also, as the columns include the prefix now, so do the metadata entries.

I use the the algorithm name as prefix for stereo features and `f"{algorithm}_tel"` as prefix for telescope-wise features. In a first iteration I used `mono`, but this is misleading at least in the case of the impact, as it is a information per telescope but derived from a stereo method.

The Container class attribute `container_prefix` was also renamed to the more fitting `default_prefix` and the prefix can now be passed to `Container.__init__`.

codecov[bot]: # [Codecov](https://codecov.io/gh/cta-observatory/ctapipe/pull/1949?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=cta-observatory) Report
> Merging [#1949](https://codecov.io/gh/cta-observatory/ctapipe/pull/1949?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=cta-observatory) (2152650) into [master](https://codecov.io/gh/cta-observatory/ctapipe/commit/cbb01b5a041725a55288b6a7da1e1c4f0541e675?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=cta-observatory) (cbb01b5) will **decrease** coverage by `0.00%`.
> The diff coverage is `66.66%`.

```diff
@@            Coverage Diff             @@
##           master    #1949      +/-   ##
==========================================
- Coverage   92.07%   92.07%   -0.01%     
==========================================
  Files         191      191              
  Lines       15088    15089       +1     
==========================================
  Hits        13893    13893              
- Misses       1195     1196       +1     
```


| [Impacted Files](https://codecov.io/gh/cta-observatory/ctapipe/pull/1949?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=cta-observatory) | Coverage Δ | |
|---|---|---|
| [ctapipe/io/tableloader.py](https://codecov.io/gh/cta-observatory/ctapipe/pull/1949/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=cta-observatory#diff-Y3RhcGlwZS9pby90YWJsZWxvYWRlci5weQ==) | `91.37% <ø> (+0.31%)` | :arrow_up: |
| [ctapipe/reco/impact.py](https://codecov.io/gh/cta-observatory/ctapipe/pull/1949/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=cta-observatory#diff-Y3RhcGlwZS9yZWNvL2ltcGFjdC5weQ==) | `44.67% <0.00%> (-0.31%)` | :arrow_down: |
| [ctapipe/reco/hillas\_intersection.py](https://codecov.io/gh/cta-observatory/ctapipe/pull/1949/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=cta-observatory#diff-Y3RhcGlwZS9yZWNvL2hpbGxhc19pbnRlcnNlY3Rpb24ucHk=) | `93.75% <100.00%> (+0.07%)` | :arrow_up: |
| [ctapipe/reco/hillas\_reconstructor.py](https://codecov.io/gh/cta-observatory/ctapipe/pull/1949/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=cta-observatory#diff-Y3RhcGlwZS9yZWNvL2hpbGxhc19yZWNvbnN0cnVjdG9yLnB5) | `98.41% <100.00%> (+0.02%)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/cta-observatory/ctapipe/pull/1949?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=cta-observatory).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=cta-observatory)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/cta-observatory/ctapipe/pull/1949?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=cta-observatory). Last update [cbb01b5...2152650](https://codecov.io/gh/cta-observatory/ctapipe/pull/1949?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=cta-observatory). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=cta-observatory).

maxnoe: Note: this breaks backwards compatibility with previously produced dl2 files.
maxnoe: Unfortunately, its not as easy as this, as we allow partial reading of the containers...


maxnoe: @kosack done

Patch:
diff --git a/ctapipe/containers.py b/ctapipe/containers.py
index 020e0950..87588e3e 100644
--- a/ctapipe/containers.py
+++ b/ctapipe/containers.py
@@ -95,7 +95,7 @@ class EventType(enum.Enum):
 class EventIndexContainer(Container):
     """index columns to include in event lists, common to all data levels"""
 
-    container_prefix = ""  # don't want to prefix these
+    default_prefix = ""  # don't want to prefix these
     obs_id = Field(0, "observation identifier")
     event_id = Field(0, "event identifier")
 
@@ -106,7 +106,7 @@ class TelEventIndexContainer(Container):
     levels that have telescope-wise information
     """
 
-    container_prefix = ""  # don't want to prefix these
+    default_prefix = ""  # don't want to prefix these
     obs_id = Field(0, "observation identifier")
     event_id = Field(0, "event identifier")
     tel_id = Field(0, "telescope identifier")
@@ -130,7 +130,7 @@ class CameraHillasParametersContainer(BaseHillasParametersContainer):
     is given in meter from the camera center.
     """
 
-    container_prefix = "camera_frame_hillas"
+    default_prefix = "camera_frame_hillas"
     x = Field(nan * u.m, "centroid x coordinate", unit=u.m)
     y = Field(nan * u.m, "centroid x coordinate", unit=u.m)
     r = Field(nan * u.m, "radial coordinate of centroid", unit=u.m)
@@ -150,7 +150,7 @@ class HillasParametersContainer(BaseHillasParametersContainer):
     longitude and latitude in degree.
     """
 
-    container_prefix = "hillas"
+    default_prefix = "hillas"
     fov_lon = Field(
         nan * u.deg,
         "longitude angle in a spherical system centered on the pointing position",
@@ -177,7 +177,7 @@ class LeakageContainer(Container):
     camera, measured in number of signal pixels or in intensity.
     """
 
-    container_prefix = "leakage"
+    default_prefix = "leakage"
 
     pixels_width_1 = Field(
         nan, "fraction of pixels after cleaning that are in camera border of width=1"
@@ -203,7 +203,7 @@ class ConcentrationContainer(Container):
     in certain areas of the image and the full image.
     """
 
-    container_prefix = "concentration"
+    default_prefix = "concentration"
     cog = Field(
         nan, "Percentage of photo-electrons inside one pixel diameter of the cog"
     )
@@ -232,7 +232,7 @@ class CameraTimingParametersContainer(BaseTimingParametersContainer):
     along the shower main axis in the camera frame.
     """
 
-    container_prefix = "camera_frame_timing"
+    default_prefix = "camera_frame_timing"
     slope = Field(
         nan / u.m, "Slope of arrival times along main shower axis", unit=1 / u.m
     )
@@ -245,7 +245,7 @@ class TimingParametersContainer(BaseTimingParametersContainer):
     spherical system centered on the pointing position (TelescopeFrame)
     """
 
-    container_prefix = "timing"
+    default_prefix = "timing"
     slope = Field(
         nan / u.deg, "Slope of arrival times along main shower axis", unit=1 / u.deg
     )
@@ -273,24 +273,24 @@ class StatisticsContainer(Container):
 
 
 class IntensityStatisticsContainer(StatisticsContainer):
-    container_prefix = "intensity"
+    default_prefix = "intensity"
 
 
 class PeakTimeStatisticsContainer(StatisticsContainer):
-    container_prefix = "peak_time"
+    default_prefix = "peak_time"
 
 
 class CoreParametersContainer(Container):
     """Telescope-wise shower's direction in the Tilted/Ground Frame"""
 
-    container_prefix = "core"
+    default_prefix = "core"
     psi = Field(nan * u.deg, "Image direction in the Tilted/Ground Frame", unit="deg")
 
 
 class ImageParametersContainer(Container):
     """Collection of image parameters"""
 
-    container_prefix = "params"
+    default_prefix = "params"
     hillas = Field(
         default_factory=HillasParametersContainer,
         description="Hillas Parameters",
@@ -497,7 +497,7 @@ class TelescopeImpactParameterContainer(Container):
     Impact Parameter computed from reconstructed shower geometry
     """
 
-    container_prefix = "impact"
+    default_prefix = "impact"
 
     distance = Field(
         nan * u.m, "distance of the telescope to the shower axis", unit=u.m
@@ -506,7 +506,7 @@ class TelescopeImpactParameterContainer(Container):
 
 
 class SimulatedShowerContainer(Container):
-    container_prefix = "true"
+    default_prefix = "true"
     energy = Field(nan * u.TeV, "Simulated Energy", unit=u.TeV)
     alt = Field(nan * u.deg, "Simulated altitude", unit=u.deg)
     az = Field(nan * u.deg, "Simulated azimuth", unit=u.deg)
@@ -530,7 +530,7 @@ class SimulatedCameraContainer(Container):
     but for simulated data.
     """
 
-    container_prefix = ""
+    default_prefix = ""
 
     true_image_sum = Field(
         np.int32(-1), "Total number of detected Cherenkov photons in the camera"
@@ -650,7 +650,7 @@ class SimulationConfigContainer(Container):
 
 
 class TelescopeTriggerContainer(Container):
-    container_prefix = ""
+    default_prefix = ""
     time = Field(NAN_TIME, description="Telescope trigger time")
     n_trigger_pixels = Field(
         -1, description="Number of trigger groups (sectors) listed"
@@ -659,7 +659,7 @@ class TelescopeTriggerContainer(Container):
 
 
 class TriggerContainer(Container):
-    container_prefix = ""
+    default_prefix = ""
     time = Field(NAN_TIME, description="central average time stamp")
     tels_with_trigger = Field(
         None, description="List of telescope ids that triggered the array event"
@@ -676,7 +676,7 @@ class ReconstructedGeometryContainer(Container):
     Standard output of algorithms reconstructing shower geometry
     """
 
-    container_prefix = ""
+    default_prefix = ""
 
     alt = Field(nan * u.deg, "reconstructed altitude", unit=u.deg)
     alt_uncert = Field(nan * u.deg, "reconstructed altitude uncertainty", unit=u.deg)
@@ -735,7 +735,7 @@ class ReconstructedEnergyContainer(Container):
     Standard output of algorithms estimating energy
     """
 
-    container_prefix = ""
+    default_prefix = ""
 
     energy = Field(nan * u.TeV, "reconstructed energy", unit=u.TeV)
     energy_uncert = Field(nan * u.TeV, "reconstructed energy uncertainty", unit=u.TeV)
@@ -756,7 +756,7 @@ class ParticleClassificationContainer(Container):
     Standard output of gamma/hadron classification algorithms
     """
 
-    container_prefix = ""
+    default_prefix = ""
 
     # TODO: Do people agree on this? This is very MAGIC-like.
     # TODO: Perhaps an integer classification to support different classes?
@@ -1091,7 +1091,7 @@ class SimulatedShowerDistribution(Container):
     core distance.
     """
 
-    container_prefix = ""
+    default_prefix = ""
 
     obs_id = Field(-1, description="links to which events this corresponds to")
     hist_id = Field(-1, description="Histogram ID")
diff --git a/ctapipe/core/container.py b/ctapipe/core/container.py
index 77818e58..97fb8e91 100644
--- a/ctapipe/core/container.py
+++ b/ctapipe/core/container.py
@@ -260,8 +260,8 @@ class ContainerMeta(type):
         new_cls = type.__new__(cls, name, bases, dct)
 
         # if prefix was not set as a class variable, build a default one
-        if "container_prefix" not in dct:
-            new_cls.container_prefix = name.lower().replace("container", "")
+        if "default_prefix" not in dct:
+            new_cls.default_prefix = name.lower().replace("container", "")
 
         return new_cls
 
@@ -317,12 +317,12 @@ class Container(metaclass=ContainerMeta):
 
     """
 
-    def __init__(self, **fields):
+    def __init__(self, prefix=None, **fields):
         self.meta = {}
         # __slots__ cannot be provided with defaults
-        # via class variables, so we use a `container_prefix` class variable
+        # via class variables, so we use a `default_prefix` class variable
         # and an instance variable `prefix` in `__slots__`
-        self.prefix = self.container_prefix
+        self.prefix = prefix if prefix is not None else self.default_prefix
 
         for k in set(self.fields).difference(fields):
 
diff --git a/ctapipe/image/cleaning.py b/ctapipe/image/cleaning.py
index 78fdde8e..eeace737 100644
--- a/ctapipe/image/cleaning.py
+++ b/ctapipe/image/cleaning.py
@@ -18,8 +18,6 @@ __all__ = [
     "mars_cleaning_1st_pass",
     "fact_image_cleaning",
     "apply_time_delta_cleaning",
-    "apply_time_average_cleaning",
-    "time_constrained_clean",
     "ImageCleaner",
     "TailcutsImageCleaner",
 ]
@@ -34,7 +32,6 @@ from ..core.traits import (
     IntTelescopeParameter,
     BoolTelescopeParameter,
 )
-from .morphology import number_of_islands, brightest_island
 
 
 def tailcuts_clean(
@@ -241,54 +238,6 @@ def apply_time_delta_cleaning(
     return pixels_to_keep
 
 
-def apply_time_average_cleaning(
-    geom, mask, image, arrival_times, picture_thresh, time_limit
-):
-    """
-    Extract all pixels that arrived within a given timeframe
-    with respect to the time average of the pixels on the main island.
-
-    In order to avoid removing signal pixels of large impact-parameter events,
-    the time limit for bright pixels is doubled.
-
-    Parameters
-    ----------
-    geom: `ctapipe.instrument.CameraGeometry`
-        Camera geometry information
-    mask: array, boolean
-        boolean mask of *clean* pixels before time_delta_cleaning
-    image: array
-        pixel values
-    arrival_times: array
-        pixel timing information
-    picture_thresh: float
-        threshold above which time limit is extended twice its value
-    time_limit: int or float
-        arrival time limit w.r.t. the average time of the core pixels
-
-    Returns
-    -------
-
-    A boolean mask of *clean* pixels.
-    """
-    mask = mask.copy()
-    if np.count_nonzero(mask) > 0:
-
-        # use main island (maximum charge) for time average calculation
-        num_islands, island_labels = number_of_islands(geom, mask)
-        mask_main = brightest_island(num_islands, island_labels, image)
-        time_ave = np.average(arrival_times[mask_main], weights=image[mask_main] ** 2)
-
-        time_diffs = np.abs(arrival_times[mask] - time_ave)
-        time_limit_pixwise = np.where(
-            image < (2 * picture_thresh), time_limit, time_limit * 2
-        )[mask]
-
-        mask[mask] &= time_diffs < time_limit_pixwise
-
-    return mask
-
-
 def fact_image_cleaning(
     geom,
     image,
@@ -374,92 +323,6 @@ def fact_image_cleaning(
     return pixels_to_keep
 
 
-def time_constrained_clean(
-    geom,
-    image,
-    arrival_times,
-    picture_thresh=7,
-    boundary_thresh=5,
-    time_limit_core=4.5,
-    time_limit_boundary=1.5,
-    min_number_picture_neighbors=1,
-):
-    """
-    time constrained cleaning by MAGIC
-
-    Cleaning contains the following steps:
-    - Find core pixels (containing more photons than a picture threshold)
-    - Remove pixels with less than N neighbors
-    - Remove pixels whose arrival times are within a time limit of the average time
-    - Find boundary pixels (containing more photons than a boundary threshold)
-    - Remove pixels with less than N neighbors arriving within a given timeframe
-
-    Parameters
-    ----------
-    geom: `ctapipe.instrument.CameraGeometry`
-        Camera geometry information
-    image: array
-        pixel values
-    arrival_times: array
-        pixel timing information
-    picture_threshold: float or array
-        threshold above which all pixels are retained
-    boundary_threshold: float or array
-        threshold above which pixels are retained if they have a neighbor
-        already above the picture_thresh
-    time_limit_core: int or float
-        arrival time limit of core pixels w.r.t the average time
-    time_limit_boundary: int or float
-        arrival time limit of boundary pixels w.r.t neighboring core pixels
-    min_number_neighbors: int
-        Threshold to determine if a pixel survives cleaning steps.
-        These steps include checks of neighbor arrival time and value
-
-    Returns
-    -------
-    A boolean mask of *clean* pixels.
-    """
-
-    # find core pixels that pass a picture threshold
-    pixels_above_picture = image >= picture_thresh
-
-    # require at least min_number_picture_neighbors
-    number_of_neighbors_above_picture = geom.neighbor_matrix_sparse.dot(
-        pixels_above_picture.view(np.byte)
-    )
-    pixels_in_picture = pixels_above_picture & (
-        number_of_neighbors_above_picture >= min_number_picture_neighbors
-    )
-
-    # keep core pixels whose arrival times are within a certain time limit of the average
-    mask_core = apply_time_average_cleaning(
-        geom, pixels_in_picture, image, arrival_times, picture_thresh, time_limit_core
-    )
-
-    # find boundary pixels that pass a boundary threshold
-    pixels_above_boundary = image >= boundary_thresh
-    pixels_with_picture_neighbors = geom.neighbor_matrix_sparse.dot(mask_core)
-    mask_boundary = (pixels_above_boundary & pixels_with_picture_neighbors) & np.invert(
-        mask_core
-    )
-
-    # keep boundary pixels whose arrival times are within a certain time limit of the neighboring core pixels
-    mask_boundary = mask_boundary.copy()
-
-    time_diffs = np.abs(arrival_times[mask_boundary, None] - arrival_times)
-    valid_neighbors = (
-        (time_diffs < time_limit_boundary)
-        & geom.neighbor_matrix[mask_boundary]
-        & mask_core
-    )
-    enough_neighbors = (
-        np.count_nonzero(valid_neighbors, axis=1) >= min_number_picture_neighbors
-    )
-    mask_boundary[mask_boundary] &= enough_neighbors
-
-    return mask_core | mask_boundary
-
-
 class ImageCleaner(TelescopeComponent):
     """
     Abstract class for all configurable Image Cleaning algorithms.   Use
@@ -567,7 +430,7 @@ class FACTImageCleaner(TailcutsImageCleaner):
     def __call__(
         self, tel_id: int, image: np.ndarray, arrival_times=None
     ) -> np.ndarray:
-        """Apply FACT-style image cleaning. see ImageCleaner.__call__()"""
+        """ Apply FACT-style image cleaning. see ImageCleaner.__call__()"""
         return fact_image_cleaning(
             geom=self.subarray.tel[tel_id].camera.geometry,
             image=image,
@@ -577,36 +440,3 @@ class FACTImageCleaner(TailcutsImageCleaner):
             min_number_neighbors=self.min_picture_neighbors.tel[tel_id],
             time_limit=self.time_limit_ns.tel[tel_id],
         )
-
-
-class TimeConstrainedImageCleaner(TailcutsImageCleaner):
-    """
-    MAGIC-like Image cleaner with timing information (See `ctapipe.image.time_constrained_clean`)
-    """
-
-    time_limit_core_ns = FloatTelescopeParameter(
-        default_value=4.5,
-        help="arrival time limit for neighboring " "core pixels, in ns",
-    ).tag(config=True)
-    time_limit_boundary_ns = FloatTelescopeParameter(
-        default_value=1.5,
-        help="arrival time limit for neighboring " "boundary pixels, in ns",
-    ).tag(config=True)
-
-    def __call__(
-        self, tel_id: int, image: np.ndarray, arrival_times=None
-    ) -> np.ndarray:
-        """
-        Apply MAGIC-like image cleaning with timing information. See `ImageCleaner.__call__()`
-        """
-
-        return time_constrained_clean(
-            self.subarray.tel[tel_id].camera.geometry,
-            image,
-            arrival_times=arrival_times,
-            picture_thresh=self.picture_threshold_pe.tel[tel_id],
-            boundary_thresh=self.boundary_threshold_pe.tel[tel_id],
-            min_number_picture_neighbors=self.min_picture_neighbors.tel[tel_id],
-            time_limit_core=self.time_limit_core_ns.tel[tel_id],
-            time_limit_boundary=self.time_limit_boundary_ns.tel[tel_id],
-        )
diff --git a/ctapipe/image/image_processor.py b/ctapipe/image/image_processor.py
index 81effd10..94450dda 100644
--- a/ctapipe/image/image_processor.py
+++ b/ctapipe/image/image_processor.py
@@ -42,12 +42,10 @@ DEFAULT_PEAKTIME_STATISTICS = PeakTimeStatisticsContainer()
 
 
 class ImageQualityQuery(QualityQuery):
-    """ for configuring image-wise data checks """
+    """for configuring image-wise data checks"""
 
     quality_criteria = List(
-        default_value=[
-            ("size_greater_0", "image.sum() > 0")
-        ],
+        default_value=[("size_greater_0", "image.sum() > 0")],
         help=QualityQuery.quality_criteria.help,
     ).tag(config=True)
 
@@ -68,8 +66,7 @@ class ImageProcessor(TelescopeComponent):
     ).tag(config=True)
 
     apply_image_modifier = BoolTelescopeParameter(
-        default_value=False,
-        help="If true, apply ImageModifier to dl1 images"
+        default_value=False, help="If true, apply ImageModifier to dl1 images"
     ).tag(config=True)
 
     def __init__(
@@ -236,6 +233,10 @@ class ImageProcessor(TelescopeComponent):
                     peak_time=None,  # true image from simulation has no peak time
                     default=DEFAULT_TRUE_IMAGE_PARAMETERS,
                 )
+                for container in sim_camera.true_parameters.values():
+                    if not container.prefix.startswith("true_"):
+                        container.prefix = f"true_{container.prefix}"
+
                 self.log.debug(
                     "sim params: %s",
                     event.simulation.tel[tel_id].true_parameters.as_dict(
diff --git a/ctapipe/io/datawriter.py b/ctapipe/io/datawriter.py
index 1a4a2fa5..5b773c50 100644
--- a/ctapipe/io/datawriter.py
+++ b/ctapipe/io/datawriter.py
@@ -46,8 +46,11 @@ def _get_tel_index(event, tel_id):
 #   (meaning readers need to update scripts)
 # - increase the minor number if new columns or datasets are added
 # - increase the patch number if there is a small bugfix to the model.
-DATA_MODEL_VERSION = "v3.0.0"
+DATA_MODEL_VERSION = "v4.0.0"
 DATA_MODEL_CHANGE_HISTORY = """
+- v4.0.0: Container prefixes are now included for reconstruction algorithms
+          and true parameters.
+          Telescope Impact Parameters were added.
 - v3.0.0: reconstructed core uncertainties splitted in their X-Y components
 - v2.2.0: added R0 and R1 outputs
 - v2.1.0: hillas and timing parameters are per default saved in telescope frame (degree) as opposed to camera frame (m)
@@ -423,6 +426,8 @@ class DataWriter(Component):
         writer.exclude("/dl1/monitoring/telescope/pointing/.*", "n_trigger_pixels")
         writer.exclude("/dl1/monitoring/telescope/pointing/.*", "trigger_pixels")
         writer.exclude("/dl1/monitoring/event/pointing/.*", "event_type")
+        writer.exclude("/dl1/event/telescope/images/.*", "parameters")
+        writer.exclude("/simulation/event/telescope/images/.*", "true_parameters")
 
         if not self.write_images:
             writer.exclude("/simulation/event/telescope/images/.*", "true_image")
@@ -430,11 +435,6 @@ class DataWriter(Component):
         if not self.write_parameters:
             writer.exclude("/dl1/event/telescope/images/.*", "image_mask")
 
-        if self._is_simulation:
-            # no timing information yet for true images
-            writer.exclude("/simulation/event/telescope/parameters/.*", r"peak_time_.*")
-            writer.exclude("/simulation/event/telescope/parameters/.*", "timing_.*")
-
         # Set up transforms
         if self.transform_image:
             transform = FixedPointColumnTransform(
@@ -472,10 +472,10 @@ class DataWriter(Component):
         # set up DL2 transforms:
         # - the single-tel output has no list of tel_ids
         # - the stereo output tel_ids list needs to be transformed to a pattern
-        writer.exclude("dl2/event/telescope/.*", "tel_ids")
+        writer.exclude("dl2/event/telescope/.*", ".*_tel_ids")
         writer.add_column_transform_regexp(
             table_regexp="dl2/event/subarray/.*",
-            col_regexp="tel_ids",
+            col_regexp=".*_tel_ids",
             transform=tr_tel_list_to_mask,
         )
 
@@ -504,7 +504,7 @@ class DataWriter(Component):
         class ExtraSimInfo(Container):
             """just to contain obs_id"""
 
-            container_prefix = ""
+            default_prefix = ""
             obs_id = Field(0, "Simulation Run Identifier")
 
         for obs_id, config in self.event_source.simulation_config.items():
@@ -666,15 +666,21 @@ class DataWriter(Component):
                     and event.simulation.tel[tel_id].true_image is not None
                 )
                 if self.write_parameters and has_sim_image:
+                    true_parameters = event.simulation.tel[tel_id].true_parameters
+                    # only write the available containers, no peak time related
+                    # features for true image available.
                     writer.write(
                         f"simulation/event/telescope/parameters/{table_name}",
                         [
                             tel_index,
-                            *event.simulation.tel[tel_id].true_parameters.values()
-                        ]
+                            true_parameters.hillas,
+                            true_parameters.leakage,
+                            true_parameters.concentration,
+                            true_parameters.morphology,
+                            true_parameters.intensity_statistics,
+                        ],
                     )
 
-
     def _write_dl2_telescope_events(
         self, writer: TableWriter, event: ArrayEventContainer
     ):
diff --git a/ctapipe/io/hdf5eventsource.py b/ctapipe/io/hdf5eventsource.py
index 81cd64a8..65348667 100644
--- a/ctapipe/io/hdf5eventsource.py
+++ b/ctapipe/io/hdf5eventsource.py
@@ -54,7 +54,7 @@ DL2_CONTAINERS = {
 }
 
 
-COMPATIBLE_DL1_VERSIONS = [
+COMPATIBLE_DATA_MODEL_VERSIONS = [
     "v1.0.0",
     "v1.0.1",
     "v1.0.2",
@@ -65,6 +65,7 @@ COMPATIBLE_DL1_VERSIONS = [
     "v2.1.0",
     "v2.2.0",
     "v3.0.0",
+    "v4.0.0",
 ]
 
 
@@ -181,10 +182,10 @@ class HDF5EventSource(EventSource):
                 return False
 
             version = metadata["CTA PRODUCT DATA MODEL VERSION"]
-            if version not in COMPATIBLE_DL1_VERSIONS:
+            if version not in COMPATIBLE_DATA_MODEL_VERSIONS:
                 logger.error(
                     f"File is DL1 file but has unsupported version {version}"
-                    f", supported versions are {COMPATIBLE_DL1_VERSIONS}"
+                    f", supported versions are {COMPATIBLE_DATA_MODEL_VERSIONS}"
                 )
                 return False
 
@@ -246,7 +247,7 @@ class HDF5EventSource(EventSource):
         # Alternatively this becomes a flat list
         # and the obs_id matching part needs to be done in _generate_events()
         class ObsIdContainer(Container):
-            container_prefix = ""
+            default_prefix = ""
             obs_id = Field(-1)
 
         simulation_configs = {}
@@ -275,17 +276,11 @@ class HDF5EventSource(EventSource):
             }
 
         if DataLevel.DL1_IMAGES in self.datalevels:
-            ignore_columns = {"parameters"}
-
-            # if there are no parameters, there are no image_mask, avoids warnings
-            if DataLevel.DL1_PARAMETERS not in self.datalevels:
-                ignore_columns.add("image_mask")
-
             image_readers = {
                 tel.name: self.reader.read(
                     f"/dl1/event/telescope/images/{tel.name}",
                     DL1CameraContainer,
-                    ignore_columns=ignore_columns,
+                    ignore_columns={"parameters"},
                 )
                 for tel in self.file_.root.dl1.event.telescope.images
             }
@@ -341,7 +336,13 @@ class HDF5EventSource(EventSource):
                             MorphologyContainer,
                             IntensityStatisticsContainer,
                         ],
-                        prefixes=True,
+                        prefixes=[
+                            "true_hillas",
+                            "true_leakage",
+                            "true_concentration",
+                            "true_morphology",
+                            "true_intensity",
+                        ],
                     )
                     for tel in self.file_.root.dl1.event.telescope.parameters
                 }
@@ -362,6 +363,7 @@ class HDF5EventSource(EventSource):
                     algorithm: HDF5TableReader(self.file_).read(
                         table._v_pathname,
                         containers=container,
+                        prefixes=(algorithm,),
                     )
                     for algorithm, table in group._v_children.items()
                 }
@@ -378,12 +380,12 @@ class HDF5EventSource(EventSource):
                     continue
 
                 dl2_tel_readers[kind] = {}
-                for name, algorithm_group in group._v_children.items():
-                    dl2_tel_readers[kind][name] = {
+                for algorithm, algorithm_group in group._v_children.items():
+                    dl2_tel_readers[kind][algorithm] = {
                         key: HDF5TableReader(self.file_).read(
                             table._v_pathname,
                             containers=container,
-                            prefixes=True,
+                            prefixes=(f"{algorithm}_tel",),
                         )
                         for key, table in algorithm_group._v_children.items()
                     }
@@ -507,7 +509,7 @@ class HDF5EventSource(EventSource):
                         simulated.true_image = simulated_image_row["true_image"]
 
                 if DataLevel.DL1_PARAMETERS in self.datalevels:
-                    if f"tel_{tel_id:03d}" not in param_readers.keys():
+                    if f"tel_{tel_id:03d}" not in param_readers:
                         logger.debug(
                             f"Triggered telescope {tel_id} is missing "
                             "from the parameters table."
@@ -527,7 +529,7 @@ class HDF5EventSource(EventSource):
                         peak_time_statistics=params[6],
                     )
                     if self.has_simulated_dl1:
-                        if f"tel_{tel_id:03d}" not in param_readers.keys():
+                        if f"tel_{tel_id:03d}" not in simulated_param_readers:
                             logger.debug(
                                 f"Triggered telescope {tel_id} is missing "
                                 "from the simulated parameters table, but was "
diff --git a/ctapipe/io/hdf5tableio.py b/ctapipe/io/hdf5tableio.py
index 4c6bb637..11d54cd0 100644
--- a/ctapipe/io/hdf5tableio.py
+++ b/ctapipe/io/hdf5tableio.py
@@ -566,7 +566,7 @@ class HDF5TableReader(TableReader):
         if prefixes is False:
             prefixes = ["" for _ in containers]
         elif prefixes is True:
-            prefixes = [container.container_prefix for container in containers]
+            prefixes = [container.default_prefix for container in containers]
         elif isinstance(prefixes, str):
             prefixes = [prefixes for _ in containers]
 
@@ -606,7 +606,7 @@ class HDF5TableReader(TableReader):
                 for fieldname in missing_cols:
                     kwargs[fieldname] = None
 
-                container = cls(**kwargs)
+                container = cls(**kwargs, prefix=prefix)
                 container.meta = self._meta[table_name]
                 ret.append(container)
 
diff --git a/ctapipe/io/simteleventsource.py b/ctapipe/io/simteleventsource.py
index d845feec..ec04d91e 100644
--- a/ctapipe/io/simteleventsource.py
+++ b/ctapipe/io/simteleventsource.py
@@ -468,11 +468,12 @@ class SimTelEventSource(EventSource):
                                 self.subarray.tel_index_array[tel_id]
                             ],
                             distance_uncert=0 * u.m,
+                            prefix="true_impact",
                         )
                     else:
-                        impact_container = TelescopeImpactParameterContainer()
-
-                    impact_container.prefix = "true_impact"
+                        impact_container = TelescopeImpactParameterContainer(
+                            prefix="true_impact",
+                        )
 
                     data.simulation.tel[tel_id] = SimulatedCameraContainer(
                         true_image_sum=true_image_sums[
diff --git a/ctapipe/io/tableloader.py b/ctapipe/io/tableloader.py
index 39fe12c6..e48eec5a 100644
--- a/ctapipe/io/tableloader.py
+++ b/ctapipe/io/tableloader.py
@@ -117,14 +117,6 @@ def _get_structure(h5file):
     return "by_type"
 
 
-def _add_column_prefix(table, prefix, ignore=()):
-    """
-    Add a prefix to all columns in table besides columns in ``ignore``.
-    """
-    for col in set(table.colnames) - set(ignore):
-        table.rename_column(col, f"{prefix}_{col}")
-
-
 def _join_subarray_events(table1, table2):
     """Outer join two tables on the telescope subarray keys"""
     return join_allow_empty(table1, table2, SUBARRAY_EVENT_KEYS, "left")
@@ -318,12 +310,6 @@ class TableLoader(Component):
                             start=start,
                             stop=stop,
                         )
-
-                        # add the algorithm as prefix to distinguish multiple
-                        # algorithms predicting the same quantities
-                        _add_column_prefix(
-                            dl2, prefix=algorithm, ignore=SUBARRAY_EVENT_KEYS
-                        )
                         table = _join_subarray_events(table, dl2)
 
         if keep_order:
@@ -393,12 +379,6 @@ class TableLoader(Component):
                         dl2 = self._read_telescope_table(
                             path, tel_id, start=start, stop=stop
                         )
-
-                        # add the algorithm as prefix to distinguish multiple
-                        # algorithms predicting the same quantities
-                        _add_column_prefix(
-                            dl2, prefix=algorithm, ignore=TELESCOPE_EVENT_KEYS
-                        )
                         table = _join_telescope_events(table, dl2)
 
         if self.load_true_images:
@@ -411,7 +391,6 @@ class TableLoader(Component):
             true_parameters = self._read_telescope_table(
                 TRUE_PARAMETERS_GROUP, tel_id, start=start, stop=stop
             )
-            _add_column_prefix(true_parameters, "true", ignore=TELESCOPE_EVENT_KEYS)
             table = _join_telescope_events(table, true_parameters)
 
         if self.load_instrument:
diff --git a/ctapipe/reco/hillas_intersection.py b/ctapipe/reco/hillas_intersection.py
index 84632aae..edc4f31b 100644
--- a/ctapipe/reco/hillas_intersection.py
+++ b/ctapipe/reco/hillas_intersection.py
@@ -39,7 +39,10 @@ from ctapipe.core import traits
 __all__ = ["HillasIntersection"]
 
 
-INVALID = ReconstructedGeometryContainer(tel_ids=[])
+INVALID = ReconstructedGeometryContainer(
+    tel_ids=[],
+    prefix="HillasIntersection",
+)
 
 
 class HillasIntersection(Reconstructor):
@@ -240,7 +243,7 @@ class HillasIntersection(Reconstructor):
 
         src_error = np.sqrt(err_fov_lon**2 + err_fov_lat**2)
 
-        result = ReconstructedGeometryContainer(
+        return ReconstructedGeometryContainer(
             alt=sky_pos.altaz.alt.to(u.rad),
             az=sky_pos.altaz.az.to(u.rad),
             core_x=grd.x,
@@ -257,8 +260,8 @@ class HillasIntersection(Reconstructor):
             h_max=x_max,
             h_max_uncert=u.Quantity(np.nan * x_max.unit),
             goodness_of_fit=np.nan,
+            prefix=self.__class__.__name__,
         )
-        return result
 
     def reconstruct_nominal(self, hillas_parameters):
         """
diff --git a/ctapipe/reco/hillas_reconstructor.py b/ctapipe/reco/hillas_reconstructor.py
index f9156bb5..221a7915 100644
--- a/ctapipe/reco/hillas_reconstructor.py
+++ b/ctapipe/reco/hillas_reconstructor.py
@@ -37,7 +37,10 @@ from astropy import units as u
 __all__ = ["HillasPlane", "HillasReconstructor"]
 
 
-INVALID = ReconstructedGeometryContainer(tel_ids=[])
+INVALID = ReconstructedGeometryContainer(
+    tel_ids=[],
+    prefix="HillasReconstructor",
+)
 
 
 def angle(v1, v2):
@@ -265,26 +268,22 @@ class HillasReconstructor(Reconstructor):
         # estimate max height of shower
         h_max = self.estimate_h_max(hillas_planes)
 
-        # astropy's coordinates system rotates counter-clockwise.
-        # Apparently we assume it to be clockwise.
-        # that's why lon get's a sign
-        result = ReconstructedGeometryContainer(
+        return ReconstructedGeometryContainer(
             alt=lat,
-            az=-lon,
+            az=-lon,  # az is clockwise, lon counter-clockwise
             core_x=core_pos_ground.x,
             core_y=core_pos_ground.y,
             core_tilted_x=core_pos_tilted.x,
             core_tilted_y=core_pos_tilted.y,
-            tel_ids=[h for h in hillas_dict.keys()],
+            tel_ids=[tel_id for tel_id in hillas_dict.keys()],
             average_intensity=np.mean([h.intensity for h in hillas_dict.values()]),
             is_valid=True,
             alt_uncert=err_est_dir,
             az_uncert=err_est_dir,
             h_max=h_max,
+            prefix=self.__class__.__name__,
         )
 
-        return result
-
     def initialize_hillas_planes(
         self, hillas_dict, subarray, telescopes_pointings, array_pointing
     ):
diff --git a/ctapipe/reco/impact.py b/ctapipe/reco/impact.py
index f3e76d42..700e91e4 100644
--- a/ctapipe/reco/impact.py
+++ b/ctapipe/reco/impact.py
@@ -395,7 +395,7 @@ class ImPACTReconstructor(Component):
         # everything in the correct units when loading in the class
         # and ignore them from then on
 
-        zenith = (np.pi / 2) - self.array_direction.alt.to(u.rad).value
+        zenith = self.array_direction.zen.to_value(u.rad)
 
         # Geometrically calculate the depth of maximum given this test position
         x_max = self.get_shower_max(source_x, source_y, core_x, core_y, zenith)
@@ -693,7 +693,7 @@ class ImPACTReconstructor(Component):
         )
         tilt_x = tilted.x.to(u.m).value
         tilt_y = tilted.y.to(u.m).value
-        zenith = 90 * u.deg - self.array_direction.alt
+        zenith = self.array_direction.zen
 
         seeds = spread_line_seed(
             self.hillas_parameters,
@@ -716,7 +716,6 @@ class ImPACTReconstructor(Component):
         )
 
         # Create a container class for reconstructed shower
-        shower_result = ReconstructedGeometryContainer()
 
         # Convert the best fits direction and core to Horizon and ground systems and
         # copy to the shower container
@@ -727,7 +726,6 @@ class ImPACTReconstructor(Component):
         )
         horizon = nominal.transform_to(AltAz())
 
-        shower_result.alt, shower_result.az = horizon.alt, horizon.az
         tilted = TiltedGroundFrame(
             x=fit_params[2] * u.m,
             y=fit_params[3] * u.m,
@@ -736,36 +734,41 @@ class ImPACTReconstructor(Component):
         )
         ground = project_to_ground(tilted)
 
-        shower_result.core_x = ground.x
-        shower_result.core_y = ground.y
-
-        shower_result.is_valid = True
-
-        # Currently no errors not available to copy NaN
-        shower_result.alt_uncert = np.nan
-        shower_result.az_uncert = np.nan
-        shower_result.core_uncert = np.nan
-
-        # Copy reconstructed Xmax
-        shower_result.h_max = fit_params[5] * self.get_shower_max(
-            fit_params[0],
-            fit_params[1],
-            fit_params[2],
-            fit_params[3],
-            zenith.to(u.rad).value,
+        h_max = (
+            fit_params[5]
+            * np.cos(zenith)
+            * self.get_shower_max(
+                fit_params[0],
+                fit_params[1],
+                fit_params[2],
+                fit_params[3],
+                zenith.to(u.rad).value,
+            )
         )
 
-        shower_result.h_max *= np.cos(zenith)
-        shower_result.h_max_uncert = errors[5] * shower_result.h_max
-
-        shower_result.goodness_of_fit = like
+        shower_result = ReconstructedGeometryContainer(
+            alt=horizon.alt,
+            az=horizon.az,
+            core_x=ground.x,
+            core_y=ground.y,
+            is_valid=True,
+            alt_uncert=np.nan,
+            az_uncert=np.nan,
+            core_uncert=np.nan,
+            # Copy reconstructed hmax
+            h_max=h_max,
+            h_max_uncert=errors[5] * h_max,
+            goodness_of_fit=like,
+            prefix=self.__class__.__name__,
+        )
 
         # Create a container class for reconstructed energy
-        energy_result = ReconstructedEnergyContainer()
-        # Fill with results
-        energy_result.energy = fit_params[4] * u.TeV
-        energy_result.energy_uncert = errors[4] * u.TeV
-        energy_result.is_valid = True
+        energy_result = ReconstructedEnergyContainer(
+            prefix=self.__class__.__name__,
+            energy=fit_params[4] * u.TeV,
+            energy_uncert=errors[4] * u.TeV,
+            is_valid=True,
+        )
 
         return shower_result, energy_result
 
diff --git a/ctapipe/reco/shower_processor.py b/ctapipe/reco/shower_processor.py
index 8ed30b6c..45a03438 100644
--- a/ctapipe/reco/shower_processor.py
+++ b/ctapipe/reco/shower_processor.py
@@ -85,5 +85,6 @@ class ShowerProcessor(Component):
         for tel_id in event.trigger.tels_with_trigger:
             tel_index = self.subarray.tel_indices[tel_id]
             event.dl2.tel[tel_id].impact[k] = TelescopeImpactParameterContainer(
-                distance=impact_distances[tel_index]
+                distance=impact_distances[tel_index],
+                prefix=f"{self.reconstructor_type}_tel",
             )


Test patch:
diff --git a/ctapipe/core/tests/test_container.py b/ctapipe/core/tests/test_container.py
index 29abf439..3ab3e328 100644
--- a/ctapipe/core/tests/test_container.py
+++ b/ctapipe/core/tests/test_container.py
@@ -11,34 +11,32 @@ def test_prefix():
         pass
 
     # make sure the default prefix is class name without container
-    assert AwesomeContainer.container_prefix == "awesome"
+    assert AwesomeContainer.default_prefix == "awesome"
     assert AwesomeContainer().prefix == "awesome"
 
     # make sure we can set the class level prefix at definition time
     class ReallyAwesomeContainer(Container):
-        container_prefix = "test"
+        default_prefix = "test"
 
-    assert ReallyAwesomeContainer.container_prefix == "test"
+    assert ReallyAwesomeContainer.default_prefix == "test"
     r = ReallyAwesomeContainer()
     assert r.prefix == "test"
 
-    ReallyAwesomeContainer.container_prefix = "test2"
-    # new instance should have the old prefix, old instance
-    # the one it was created with
+    # new instance should have the new prefix,
+    # old instance the one it was created with
+    ReallyAwesomeContainer.default_prefix = "test2"
     assert ReallyAwesomeContainer().prefix == "test2"
     assert r.prefix == "test"
 
-    # Make sure we can set the class level prefix at runtime
-    ReallyAwesomeContainer.container_prefix = "foo"
-    assert ReallyAwesomeContainer().prefix == "foo"
-
     # make sure we can assign instance level prefixes
     c1 = ReallyAwesomeContainer()
     c2 = ReallyAwesomeContainer()
+    c3 = ReallyAwesomeContainer(prefix="c3")
     c2.prefix = "c2"
 
-    assert c1.prefix == "foo"
+    assert c1.prefix == "test2"
     assert c2.prefix == "c2"
+    assert c3.prefix == "c3"
 
 
 def test_inheritance():
diff --git a/ctapipe/image/tests/test_cleaning.py b/ctapipe/image/tests/test_cleaning.py
index d004f7bc..eb921f8b 100644
--- a/ctapipe/image/tests/test_cleaning.py
+++ b/ctapipe/image/tests/test_cleaning.py
@@ -198,7 +198,7 @@ def test_tailcuts_clean_min_neighbors_1():
 
 
 def test_tailcuts_clean_min_neighbors_2():
-    """requiring that picture pixels have at least two neighbors above
+    """ requiring that picture pixels have at least two neighbors above
     picture_thresh"""
 
     # start with simple 3-pixel camera
@@ -331,128 +331,3 @@ def test_apply_time_delta_cleaning():
     test_mask = mask.copy()
     test_mask[[41, 157]] = 0
     assert (test_mask == td_mask).all()
-
-
-def test_time_constrained_clean():
-    geom = CameraGeometry.from_name("LSTCam")
-    charge = np.zeros(geom.n_pixels, dtype=np.float64)
-    peak_time = np.zeros(geom.n_pixels, dtype=np.float64)
-
-    # define signal pixels and their charges/timings (1 core pixel + 6 neighboring core pixels + 12 neighboring boundary pixels)
-    core_pixel = 100
-    core_neighbors = geom.neighbors[core_pixel]
-    boundary_pixels = np.setdiff1d(
-        np.array([geom.neighbors[core_neighbor] for core_neighbor in core_neighbors]),
-        np.append(core_neighbors, core_pixel),
-    )
-    charge[core_pixel], charge[core_neighbors], charge[boundary_pixels] = 15, 10, 6
-    peak_time[core_pixel], peak_time[core_neighbors], peak_time[boundary_pixels] = (
-        18,
-        20,
-        21,
-    )
-
-    # define initial cleaning parameters
-    picture_thresh, boundary_thresh = 8, 4
-    time_limit_core, time_limit_boundary = 4.5, 1.5
-    min_number_picture_neighbors = 1
-
-    mask_signal = charge > 0
-
-    # 1. basic test
-    mask_reco = cleaning.time_constrained_clean(
-        geom,
-        charge,
-        peak_time,
-        picture_thresh=picture_thresh,
-        boundary_thresh=boundary_thresh,
-        time_limit_core=time_limit_core,
-        time_limit_boundary=time_limit_boundary,
-        min_number_picture_neighbors=min_number_picture_neighbors,
-    )
-    test_mask = mask_signal.copy()
-    assert (test_mask == mask_reco).all()
-
-    # 2. increased min_number_picture_neighbors test (here 3)
-    min_number_picture_neighbors = 3
-    mask_reco = cleaning.time_constrained_clean(
-        geom,
-        charge,
-        peak_time,
-        picture_thresh=picture_thresh,
-        boundary_thresh=boundary_thresh,
-        time_limit_core=time_limit_core,
-        time_limit_boundary=time_limit_boundary,
-        min_number_picture_neighbors=min_number_picture_neighbors,
-    )
-    ## removed pixels : boundary pixels
-    test_mask = mask_signal.copy()
-    test_mask[boundary_pixels] = 0
-    assert (test_mask == mask_reco).all()
-
-    # 3. strict time_limit_boundary test (here 0.5)
-    min_number_picture_neighbors = 1
-    time_limit_boundary = 0.5
-    mask_reco = cleaning.time_constrained_clean(
-        geom,
-        charge,
-        peak_time,
-        picture_thresh=picture_thresh,
-        boundary_thresh=boundary_thresh,
-        time_limit_core=time_limit_core,
-        time_limit_boundary=time_limit_boundary,
-        min_number_picture_neighbors=min_number_picture_neighbors,
-    )
-    ## removed pixels : boundary pixels
-    test_mask = mask_signal.copy()
-    test_mask[boundary_pixels] = 0
-    assert (test_mask == mask_reco).all()
-
-    # 4. time_limit_core test (one of core_neighbors have peak time >5 slice away from the average)
-    time_limit_boundary = 1.5
-    noise_core_neighbor = core_neighbors[0]
-    peak_time[noise_core_neighbor] = 25
-    mask_reco = cleaning.time_constrained_clean(
-        geom,
-        charge,
-        peak_time,
-        picture_thresh=picture_thresh,
-        boundary_thresh=boundary_thresh,
-        time_limit_core=time_limit_core,
-        time_limit_boundary=time_limit_boundary,
-        min_number_picture_neighbors=min_number_picture_neighbors,
-    )
-    ## removed pixels : the noise core neighbor pixel + one neighboring boundary
-    test_mask = mask_signal.copy()
-    test_mask[noise_core_neighbor] = 0
-    noise_boundary = np.setdiff1d(
-        geom.neighbors[noise_core_neighbor],
-        np.array(
-            [geom.neighbors[core_neighbor] for core_neighbor in core_neighbors[1:]]
-        ),
-    )
-    test_mask[noise_boundary] = 0
-    assert (test_mask == mask_reco).all()
-
-    # 5. time_limit_core test for brighter pixels (one of core_neighbors have peak time >5 slice away from the average)
-    charge[core_pixel], charge[core_neighbors] = 30, 20
-    mask_reco = cleaning.time_constrained_clean(
-        geom,
-        charge,
-        peak_time,
-        picture_thresh=picture_thresh,
-        boundary_thresh=boundary_thresh,
-        time_limit_core=time_limit_core,
-        time_limit_boundary=time_limit_boundary,
-        min_number_picture_neighbors=min_number_picture_neighbors,
-    )
-    ## removed pixels : one neighboring boundary to the noise core pixel
-    test_mask = mask_signal.copy()
-    noise_boundary = np.setdiff1d(
-        geom.neighbors[noise_core_neighbor],
-        np.array(
-            [geom.neighbors[core_neighbor] for core_neighbor in core_neighbors[1:]]
-        ),
-    )
-    test_mask[noise_boundary] = 0
-    assert (test_mask == mask_reco).all()
diff --git a/ctapipe/io/tests/test_datawriter.py b/ctapipe/io/tests/test_datawriter.py
index a4242b0c..71af6357 100644
--- a/ctapipe/io/tests/test_datawriter.py
+++ b/ctapipe/io/tests/test_datawriter.py
@@ -9,6 +9,11 @@ import tables
 from traitlets.config import Config
 from astropy import units as u
 from ctapipe.calib import CameraCalibrator
+from ctapipe.containers import (
+    ParticleClassificationContainer,
+    ReconstructedGeometryContainer,
+    ReconstructedEnergyContainer,
+)
 from ctapipe.instrument import SubarrayDescription
 from ctapipe.io import DataLevel, EventSource
 from ctapipe.io.datawriter import DATA_MODEL_VERSION, DataWriter
@@ -22,18 +27,40 @@ def generate_dummy_dl2(event):
 
     for algo in algos:
         for tel_id in event.dl1.tel:
-            event.dl2.tel[tel_id].geometry[algo].alt = 70 * u.deg
-            event.dl2.tel[tel_id].geometry[algo].az = 120 * u.deg
-            event.dl2.tel[tel_id].energy[algo].energy = 10 * u.TeV
-            event.dl2.tel[tel_id].classification[algo].prediction = 0.9
+            event.dl2.tel[tel_id].geometry[algo] = ReconstructedGeometryContainer(
+                alt=70 * u.deg,
+                az=120 * u.deg,
+                prefix=f"{algo}_tel",
+            )
+
+            event.dl2.tel[tel_id].energy[algo] = ReconstructedEnergyContainer(
+                energy=10 * u.TeV,
+                prefix=f"{algo}_tel",
+            )
+            event.dl2.tel[tel_id].classification[
+                algo
+            ] = ParticleClassificationContainer(
+                prediction=0.9,
+                prefix=f"{algo}_tel",
+            )
+
+        event.dl2.stereo.geometry[algo] = ReconstructedGeometryContainer(
+            alt=72 * u.deg,
+            az=121 * u.deg,
+            tel_ids=[1, 2, 4],
+            prefix=algo,
+        )
 
-        event.dl2.stereo.geometry[algo].alt = 72 * u.deg
-        event.dl2.stereo.geometry[algo].az = 121 * u.deg
-        event.dl2.stereo.geometry[algo].tel_ids = [1, 2, 4]
-        event.dl2.stereo.energy[algo].tel_ids = [1, 2, 4]
-        event.dl2.stereo.energy[algo].energy = 10 * u.TeV
-        event.dl2.stereo.classification[algo].prediction = 0.9
-        event.dl2.stereo.classification[algo].tel_ids = [1, 2, 4]
+        event.dl2.stereo.energy[algo] = ReconstructedEnergyContainer(
+            energy=10 * u.TeV,
+            tel_ids=[1, 2, 4],
+            prefix=algo,
+        )
+        event.dl2.stereo.classification[algo] = ParticleClassificationContainer(
+            prediction=0.9,
+            tel_ids=[1, 2, 4],
+            prefix=algo,
+        )
 
 
 def test_write(tmpdir: Path):
@@ -51,7 +78,7 @@ def test_write(tmpdir: Path):
         get_dataset_path("gamma_LaPalma_baseline_20Zd_180Az_prod3b_test.simtel.gz"),
         max_events=20,
         allowed_tels=[1, 2, 3, 4],
-        focal_length_choice='nominal',
+        focal_length_choice="nominal",
     )
     calibrate = CameraCalibrator(subarray=source.subarray)
 
@@ -104,16 +131,17 @@ def test_write(tmpdir: Path):
             shower._v_attrs["true_alt_UNIT"] == "deg"
         )  # pylint: disable=protected-access
 
-        # check DL2:
-        dl2_energy = h5file.get_node("/dl2/event/subarray/energy/ImPACTReconstructor")
-        assert np.allclose(dl2_energy.col("energy"), 10)
-        assert np.count_nonzero(dl2_energy.col("tel_ids")[0]) == 3
+        # check DL2
+        for prefix in ("ImPACTReconstructor", "HillasReconstructor"):
+            dl2_energy = h5file.get_node(f"/dl2/event/subarray/energy/{prefix}")
+            assert np.allclose(dl2_energy.col(f"{prefix}_energy"), 10)
+            assert np.count_nonzero(dl2_energy.col(f"{prefix}_tel_ids")[0]) == 3
 
-        dl2_tel_energy = h5file.get_node(
-            "/dl2/event/telescope/energy/HillasReconstructor/tel_002"
-        )
-        assert np.allclose(dl2_tel_energy.col("energy"), 10)
-        assert "tel_ids" not in dl2_tel_energy
+            dl2_tel_energy = h5file.get_node(
+                f"/dl2/event/telescope/energy/{prefix}/tel_002"
+            )
+            assert np.allclose(dl2_tel_energy.col(f"{prefix}_tel_energy"), 10)
+            assert "tel_ids" not in dl2_tel_energy
 
 
 def test_roundtrip(tmpdir: Path):
@@ -131,7 +159,7 @@ def test_roundtrip(tmpdir: Path):
         get_dataset_path("gamma_LaPalma_baseline_20Zd_180Az_prod3b_test.simtel.gz"),
         max_events=20,
         allowed_tels=[1, 2, 3, 4],
-        focal_length_choice='nominal',
+        focal_length_choice="nominal",
     )
     calibrate = CameraCalibrator(subarray=source.subarray)
 
@@ -203,7 +231,9 @@ def test_dl1writer_no_events(tmpdir: Path):
 
     output_path = Path(tmpdir / "no_events.dl1.h5")
     dataset = "lst_prod3_calibration_and_mcphotons.simtel.zst"
-    with EventSource(get_dataset_path(dataset), focal_length_choice='nominal') as source:
+    with EventSource(
+        get_dataset_path(dataset), focal_length_choice="nominal"
+    ) as source:
         # exhaust source
         for _ in source:
             pass
diff --git a/ctapipe/io/tests/test_hdf5.py b/ctapipe/io/tests/test_hdf5.py
index b21c5aa1..77609890 100644
--- a/ctapipe/io/tests/test_hdf5.py
+++ b/ctapipe/io/tests/test_hdf5.py
@@ -571,7 +571,7 @@ def test_column_exclusions(tmp_path):
     tmp_file = tmp_path / "test_column_exclusions.hdf5"
 
     class SomeContainer(Container):
-        container_prefix = ""
+        default_prefix = ""
         hillas_x = Field(None)
         hillas_y = Field(None)
         impact_x = Field(None)
@@ -615,7 +615,7 @@ def test_column_transforms(tmp_path):
     tmp_file = tmp_path / "test_column_transforms.hdf5"
 
     class SomeContainer(Container):
-        container_prefix = ""
+        default_prefix = ""
 
         current = Field(1 * u.A, unit=u.uA)
         time = Field(NAN_TIME)
@@ -648,7 +648,7 @@ def test_fixed_point_column_transform(tmp_path):
     tmp_file = tmp_path / "test_column_transforms.hdf5"
 
     class SomeContainer(Container):
-        container_prefix = ""
+        default_prefix = ""
         image = Field(np.array([np.nan, np.inf, -np.inf]))
 
     cont = SomeContainer()
@@ -686,7 +686,7 @@ def test_column_transforms_regexps(tmp_path):
         return x * 10
 
     class SomeContainer(Container):
-        container_prefix = ""
+        default_prefix = ""
         hillas_x = Field(1)
         hillas_y = Field(1)
 
@@ -847,7 +847,7 @@ def test_strings(tmp_path):
 
     # when not giving a max_len, should be taken from the first container
     class Container1(Container):
-        container_prefix = ""
+        default_prefix = ""
         string = Field("", "test string")
 
     path = tmp_path / "test.h5"
@@ -864,7 +864,7 @@ def test_strings(tmp_path):
     assert table["string"].tolist() == ["Hello", "öä"]
 
     class Container2(Container):
-        container_prefix = ""
+        default_prefix = ""
         string = Field("", "test string", max_length=10)
 
     path = tmp_path / "test.h5"
@@ -887,3 +887,24 @@ def test_strings(tmp_path):
         for string in expected:
             c = next(generator)
             assert c.string == string
+
+
+def test_prefix_in_output_container(tmp_path):
+    """Test that output containers retain the used prefix"""
+
+    class Container1(Container):
+        default_prefix = ""
+        value = Field(-1, "value")
+
+    path = tmp_path / "prefix.h5"
+    with HDF5TableWriter(path, mode="w", add_prefix=True) as writer:
+        for value in (1, 2, 3):
+            writer.write("values", Container1(value=value, prefix="custom_prefix"))
+
+    with HDF5TableReader(path) as reader:
+        generator = reader.read("/values", Container1, prefixes="custom_prefix")
+
+        for value in (1, 2, 3):
+            c = next(generator)
+            assert c.prefix == "custom_prefix"
+            assert c.value == value
diff --git a/ctapipe/io/tests/test_hdf5eventsource.py b/ctapipe/io/tests/test_hdf5eventsource.py
index 9d792994..a4345619 100644
--- a/ctapipe/io/tests/test_hdf5eventsource.py
+++ b/ctapipe/io/tests/test_hdf5eventsource.py
@@ -159,8 +159,16 @@ def test_read_dl2(dl2_shower_geometry_file):
     with HDF5EventSource(dl2_shower_geometry_file) as s:
         e = next(iter(s))
         assert algorithm in e.dl2.stereo.geometry
+        assert e.dl2.stereo.geometry[algorithm].alt is not None
+        assert e.dl2.stereo.geometry[algorithm].az is not None
+        assert e.dl2.stereo.geometry[algorithm].tel_ids is not None
+        assert e.dl2.stereo.geometry[algorithm].prefix == algorithm
+
         tel_mask = e.dl2.stereo.geometry[algorithm].tel_ids
         tel_ids = s.subarray.tel_mask_to_tel_ids(tel_mask)
         for tel_id in tel_ids:
+            assert tel_id in e.dl2.tel
             assert algorithm in e.dl2.tel[tel_id].impact
-            assert e.dl2.tel[tel_id].impact[algorithm].distance is not None
+            impact = e.dl2.tel[tel_id].impact[algorithm]
+            assert impact.prefix == algorithm + "_tel"
+            assert impact.distance is not None
diff --git a/ctapipe/io/tests/test_table_loader.py b/ctapipe/io/tests/test_table_loader.py
index b8a63ff8..fb0e32ec 100644
--- a/ctapipe/io/tests/test_table_loader.py
+++ b/ctapipe/io/tests/test_table_loader.py
@@ -236,7 +236,7 @@ def test_read_telescope_events_type(test_file_dl2):
         expected_ids = subarray.get_tel_ids_for_type("MST_MST_FlashCam")
         assert set(table["tel_id"].data).issubset(expected_ids)
         assert "equivalent_focal_length" in table.colnames
-        assert "HillasReconstructor_impact_distance" in table.colnames
+        assert "HillasReconstructor_tel_distance" in table.colnames
 
 
 def test_read_telescope_events_by_type(test_file_dl2):
diff --git a/ctapipe/tools/tests/test_process.py b/ctapipe/tools/tests/test_process.py
index b041df41..f463ab53 100644
--- a/ctapipe/tools/tests/test_process.py
+++ b/ctapipe/tools/tests/test_process.py
@@ -4,10 +4,12 @@
 Test ctapipe-process on a few different use cases
 """
 
+import numpy as np
 import pandas as pd
 import pytest
 import tables
 from ctapipe.core import run_tool
+from ctapipe.instrument.subarray import SubarrayDescription
 from ctapipe.io import DataLevel, EventSource, read_table
 from ctapipe.tools.process import ProcessorTool
 from ctapipe.tools.quickstart import CONFIGS_TO_WRITE, QuickStartTool
@@ -124,21 +126,25 @@ def test_stage_1_dl1(tmp_path, dl1_image_file, dl1_parameters_file):
     for feature in features:
         assert feature in dl1_features.columns
 
+    true_impact = read_table(
+        dl1b_from_dl1a_file,
+        "/simulation/event/telescope/impact/tel_025",
+    )
+    assert "true_impact_distance" in true_impact.colnames
+
     # DL1B file as input
-    assert (
-        run_tool(
-            ProcessorTool(),
-            argv=[
-                f"--config={config}",
-                f"--input={dl1_parameters_file}",
-                f"--output={tmp_path}/dl1b_from_dl1b.dl1.h5",
-                "--write-parameters",
-                "--overwrite",
-            ],
-            cwd=tmp_path,
-        )
-        == 1
+    ret = run_tool(
+        ProcessorTool(),
+        argv=[
+            f"--config={config}",
+            f"--input={dl1_parameters_file}",
+            f"--output={tmp_path}/dl1b_from_dl1b.dl1.h5",
+            "--write-parameters",
+            "--overwrite",
+        ],
+        cwd=tmp_path,
     )
+    assert ret == 1
 
 
 def test_stage1_datalevels(tmp_path):
@@ -222,7 +228,16 @@ def test_stage_2_from_simtel(tmp_path):
 
     # check tables were written
     with tables.open_file(output, mode="r") as testfile:
-        assert testfile.root.dl2.event.subarray.geometry.HillasReconstructor
+        dl2 = read_table(
+            testfile,
+            "/dl2/event/subarray/geometry/HillasReconstructor",
+        )
+        subarray = SubarrayDescription.from_hdf(testfile)
+
+        # test tel_ids are included and transformed correctly
+        assert "HillasReconstructor_tel_ids" in dl2.colnames
+        assert dl2["HillasReconstructor_tel_ids"].dtype == np.bool_
+        assert dl2["HillasReconstructor_tel_ids"].shape[1] == len(subarray)
 
 
 def test_stage_2_from_dl1_images(tmp_path, dl1_image_file):