Repository: kedro-org/kedro

Issue:
DataCatalog.shallow_copy() destroys any `CustomDataCatalog` type object
## Description
kedro runner calls  data catalog `shallow_copy` that always return new `DataCatalog` object type, destroying any `CustomDataCatalog` object being copied.

## Context
I created custom `PickleDataCatalog` class in order to handle dynamically multiple pickle objects on `after_node_run` hook event.
My custom datacatalog and hooks were properly set in `src/my_project_name/settings.py`
```
from .hooks import ModelSavingHook  # noqa: F401
HOOKS = (ModelSavingHook(),)
...
from .pickle import PickleDataCatalog
DATA_CATALOG_CLASS = PickleDataCatalog
```
Turned out that preset `DATA_CATALOG_CLASS` reference is lost during pipeline lifecycle and the `catalog` param is `DataCatalog` type instead of expected custom `PickleDataCatalog`

## Steps to Reproduce
1) create simple custom `PickleDataCatalog` class
```
class PickleDataCatalog(DataCatalog):
    def __init__(self, pickle_directory : str =  'data/models/', *args, **kwargs):
        super().__init__(*args, **kwargs)
        logging.info(f"PickleDataCatalog instance created, id: {id(self)}")
        self._pickle_directory = pickle_directory

    def save(self, name: str, data: Any):
        if name not in self._datasets:
            self.add(name, PickleDataset(filepath=os.path.join(self._pickle_directory, f"{name}.pickle")))
        super().save(name, data)

```
2) create `hooks.py`with custom `ModelSavingHook` handler, to properly maintain files:
```python
class ModelSavingHook:
    """Hook to save models to disk after they are run."""

    @hook_impl
    def before_node_run(  # noqa: PLR0913
        self,
        node: Node,
        catalog: DataCatalog,
        inputs: dict[str, Any],
        is_async: bool,
        session_id: str,
    ) :
        pass # load everything that is required

    @hook_impl
    def after_node_run(self, catalog: PickleDataCatalog, outputs: Dict[str, Any], node, inputs: Dict[str, Any]):
        logging.info(f"Catalog type in hook: {type(catalog)}, id: {id(catalog)}")
        if not isinstance(catalog, PickleDataCatalog):
                raise TypeError(
                    f"Expected `PickleDataCatalog`, got {type(catalog)}"
                )
        # iterating output and saving objects to passed `catalog` param
```
3) update `src/my_project_name/settings.py`
```
from .hooks import ModelSavingHook  # noqa: F401
HOOKS = (ModelSavingHook(),)
...
from .pickle import PickleDataCatalog
DATA_CATALOG_CLASS = PickleDataCatalog
```

4) `kedro run`


## Expected Result
While investigating pipeline lifecycle, i can see that custom DataCatalog is properly propagated among following events:

* `after_catalog_created`
* `before_pipeline_run`

Unfortunatelly it is lost on `before_node_run` and  `after_node_run`, expected :
```
@hook_impl
    def before_node_run(  # noqa: PLR0913
        self,
        node: Node,
        catalog: DataCatalog,
        inputs: dict[str, Any],
        is_async: bool,
        session_id: str,
    ) :

   assert isinstance(catalog, PickleDataCatalog) == True
```

## Actual Result
Received `catalog` object  on `before_node_run` is different than on `after_catalog_created` and `before_pipeline_run`! 
instead of expected `PickleDataCatalog` there is passed default `kedro.io.DataCatalog` (check `id(catalog)}` on all events)

## Cause of a problem

Turned out in `kedro.runner` there is custom `shallow_copy` call on catalog

```python
def run(
        self,
        pipeline: Pipeline,
        catalog: DataCatalog,
        hook_manager: PluginManager | None = None,
        session_id: str | None = None,
    ) -> dict[str, Any]:
    ..
    catalog = catalog.shallow_copy()
    ...
```
where
```python
def shallow_copy(
        self, extra_dataset_patterns: Patterns | None = None
    ) -> DataCatalog:
        """Returns a shallow copy of the current object.

        Returns:
            Copy of the current object.
        """
        ...
        return DataCatalog(
            datasets=self._datasets,
            dataset_patterns=dataset_patterns,
            load_versions=self._load_versions,
            save_version=self._save_version,
        )
```

In order to fix that bug we need to use the actual type of `catalog` object - which would be:
```python
def shallow_copy(
        self, extra_dataset_patterns: Patterns | None = None
    ) -> DataCatalog:
        """Returns a shallow copy of the current object.

        Returns:
            Copy of the current object.
        """
        ...
        return self.__class__(
            datasets=self._datasets,
            dataset_patterns=dataset_patterns,
            load_versions=self._load_versions,
            save_version=self._save_version,
        )
```
## Your Environment
<!-- Include as many relevant details about the environment in which you experienced the bug: -->

* Kedro version used (`pip show kedro` or `kedro -V`): kedro, version 0.19.5
* Python version used (`python -V`): Python 3.9.17
* Operating system and version: macOS



Discussion:
Hi @tzdanowicz, thank you for creating this issue. This sounds like a bug on our end, so I'll make sure we address it and put it on our backlog. 



Patch:
diff --git a/RELEASE.md b/RELEASE.md
index 21925f4395..262d8e3792 100644
--- a/RELEASE.md
+++ b/RELEASE.md
@@ -4,6 +4,7 @@
 
 ## Bug fixes and other changes
 * Updated error message for invalid catalog entries.
+* Fixed a bug in the `DataCatalog` `shallow_copy()` method to ensure it returns the type of the used catalog and doesn't cast it to `DataCatalog`.
 
 ## Breaking changes to the API
 
diff --git a/kedro/io/data_catalog.py b/kedro/io/data_catalog.py
index 7e2bdebdf1..7778cfd70f 100644
--- a/kedro/io/data_catalog.py
+++ b/kedro/io/data_catalog.py
@@ -761,7 +761,7 @@ def shallow_copy(
             dataset_patterns = self._sort_patterns(unsorted_dataset_patterns)
         else:
             dataset_patterns = self._dataset_patterns
-        return DataCatalog(
+        return self.__class__(
             datasets=self._datasets,
             dataset_patterns=dataset_patterns,
             load_versions=self._load_versions,


Test patch:
diff --git a/tests/io/test_data_catalog.py b/tests/io/test_data_catalog.py
index 4a4caff7c6..26c3659cef 100644
--- a/tests/io/test_data_catalog.py
+++ b/tests/io/test_data_catalog.py
@@ -426,11 +426,21 @@ def test_confirm(self, mocker, caplog):
         ],
     )
     def test_bad_confirm(self, data_catalog, dataset_name, error_pattern):
-        """Test confirming a non existent dataset or one that
+        """Test confirming a non-existent dataset or one that
         does not have `confirm` method"""
         with pytest.raises(DatasetError, match=re.escape(error_pattern)):
             data_catalog.confirm(dataset_name)
 
+    def test_shallow_copy_returns_correct_class_type(
+        self,
+    ):
+        class MyDataCatalog(DataCatalog):
+            pass
+
+        data_catalog = MyDataCatalog()
+        copy = data_catalog.shallow_copy()
+        assert isinstance(copy, MyDataCatalog)
+
 
 class TestDataCatalogFromConfig:
     def test_from_sane_config(self, data_catalog_from_config, dummy_dataframe):