Repository: kedro-org/kedro

Issue:
Avoid setting warning filters globally
## Description
We should avoid unconditionally setting warning filters globally, like we do here:

https://github.com/kedro-org/kedro/blob/2f3811aeab25c25fba8476f1152651753e85ee7e/kedro/io/core.py#L25

## Context
The dozens of `pkg_resources` warnings have been bugging me forever. They can be seen everywhere: our Read the Docs builds https://readthedocs.org/projects/kedro/builds/21147135/, our tests, any `kedro run` that makes use of certain datasets, or just importing them:

```
❯ python -c "from kedro_datasets.pandas import CSVDataSet"                                                                                                                                                                                         (kedro38-dev) 
/Users/juan_cano/.micromamba/envs/kedro38-dev/lib/python3.8/site-packages/pkg_resources/__init__.py:121: DeprecationWarning: pkg_resources is deprecated as an API
  warnings.warn("pkg_resources is deprecated as an API", DeprecationWarning)
/Users/juan_cano/.micromamba/envs/kedro38-dev/lib/python3.8/site-packages/pkg_resources/__init__.py:2870: DeprecationWarning: Deprecated call to `pkg_resources.declare_namespace('google')`.
Implementing implicit namespace packages (as specified in PEP 420) is preferred to `pkg_resources.declare_namespace`. See https://setuptools.pypa.io/en/latest/references/keywords.html#keyword-namespace-packages
  declare_namespace(pkg)
/Users/juan_cano/.micromamba/envs/kedro38-dev/lib/python3.8/site-packages/pkg_resources/__init__.py:2870: DeprecationWarning: Deprecated call to `pkg_resources.declare_namespace('google.cloud')`.
Implementing implicit namespace packages (as specified in PEP 420) is preferred to `pkg_resources.declare_namespace`. See https://setuptools.pypa.io/en/latest/references/keywords.html#keyword-namespace-packages
  declare_namespace(pkg)
/Users/juan_cano/.micromamba/envs/kedro38-dev/lib/python3.8/site-packages/pkg_resources/__init__.py:2349: DeprecationWarning: Deprecated call to `pkg_resources.declare_namespace('google')`.
Implementing implicit namespace packages (as specified in PEP 420) is preferred to `pkg_resources.declare_namespace`. See https://setuptools.pypa.io/en/latest/references/keywords.html#keyword-namespace-packages
  declare_namespace(parent)
/Users/juan_cano/.micromamba/envs/kedro38-dev/lib/python3.8/site-packages/pkg_resources/__init__.py:2870: DeprecationWarning: Deprecated call to `pkg_resources.declare_namespace('google.logging')`.
Implementing implicit namespace packages (as specified in PEP 420) is preferred to `pkg_resources.declare_namespace`. See https://setuptools.pypa.io/en/latest/references/keywords.html#keyword-namespace-packages
  declare_namespace(pkg)
/Users/juan_cano/.micromamba/envs/kedro38-dev/lib/python3.8/site-packages/pkg_resources/__init__.py:2870: DeprecationWarning: Deprecated call to `pkg_resources.declare_namespace('mpl_toolkits')`.
Implementing implicit namespace packages (as specified in PEP 420) is preferred to `pkg_resources.declare_namespace`. See https://setuptools.pypa.io/en/latest/references/keywords.html#keyword-namespace-packages
  declare_namespace(pkg)
/Users/juan_cano/.micromamba/envs/kedro38-dev/lib/python3.8/site-packages/pkg_resources/__init__.py:2870: DeprecationWarning: Deprecated call to `pkg_resources.declare_namespace('ruamel')`.
Implementing implicit namespace packages (as specified in PEP 420) is preferred to `pkg_resources.declare_namespace`. See https://setuptools.pypa.io/en/latest/references/keywords.html#keyword-namespace-packages
  declare_namespace(pkg)
/Users/juan_cano/.micromamba/envs/kedro38-dev/lib/python3.8/site-packages/pkg_resources/__init__.py:2870: DeprecationWarning: Deprecated call to `pkg_resources.declare_namespace('snowflake')`.
Implementing implicit namespace packages (as specified in PEP 420) is preferred to `pkg_resources.declare_namespace`. See https://setuptools.pypa.io/en/latest/references/keywords.html#keyword-namespace-packages
  declare_namespace(pkg)
/Users/juan_cano/.micromamba/envs/kedro38-dev/lib/python3.8/site-packages/pkg_resources/__init__.py:2870: DeprecationWarning: Deprecated call to `pkg_resources.declare_namespace('sphinxcontrib')`.
Implementing implicit namespace packages (as specified in PEP 420) is preferred to `pkg_resources.declare_namespace`. See https://setuptools.pypa.io/en/latest/references/keywords.html#keyword-namespace-packages
  declare_namespace(pkg)
/Users/juan_cano/.micromamba/envs/kedro38-dev/lib/python3.8/site-packages/google/rpc/__init__.py:20: DeprecationWarning: Deprecated call to `pkg_resources.declare_namespace('google.rpc')`.
Implementing implicit namespace packages (as specified in PEP 420) is preferred to `pkg_resources.declare_namespace`. See https://setuptools.pypa.io/en/latest/references/keywords.html#keyword-namespace-packages
  pkg_resources.declare_namespace(__name__)
```

The problem is that, even if one wants to disable them, it doesn't work: nor from the CLI (with `-Wignore`) nor from the code itself. The reason is two-fold:

1. The global registry overrides any other CLI option ([as I painfully learned the hard way](https://discuss.python.org/t/how-to-silence-pkg-resources-warnings/28629/7?u=astrojuanlu))
2. The user might set up their own override, but if ours comes after, then it stands.

As a result, at the moment it's impossible to disable warnings coming from `pkg_resources`, which are hardly actionable for users.

Now, I probably get _why_ this was done: because `DeprecationWarning`s do _not_ get shown otherwise! That was the painful realization that led to [PEP 565](https://peps.python.org/pep-0565/):

> In Python 2.7 and Python 3.2, the default warning filters were updated to hide DeprecationWarning by default [...]
> However, this change has had the unfortunate side effect of making DeprecationWarning markedly less effective at its primary intended purpose: providing advance notice of breaking changes in APIs (whether in CPython, the standard library, or in third party libraries) to users of those APIs.

At the moment (Python 3.7+), the [default warning filters](https://docs.python.org/3.7/library/warnings.html#default-warning-filter) are:

```
default::DeprecationWarning:__main__
ignore::DeprecationWarning
ignore::PendingDeprecationWarning
ignore::ImportWarning
ignore::ResourceWarning
```

Which is probably why we decided to add a `warnings.simplefilter("default", DeprecationWarning)` to our code. However, this has too many side effects, and we should consider other ways.

## Possible Implementation
I think we must remove all unconditional uses of `warnings.simplefilter` for standard Python warnings. Then, to still make our deprecation warnings visible, we could explore two possibilities:

- Using `FutureWarning` instead of `DeprecationWarning`. The advantage is that `FutureWarning` is shown everywhere by default, so it would be more visible. In the old times, `FutureWarning` was supposed to be "for warnings about constructs that will change semantically in the future." ([source](https://docs.python.org/3.2/library/exceptions.html#FutureWarning)), but from Python 3.7 onwards (post PEP 565) that's not the case anymore.
- Creating a `KedroDeprecationWarning` class, inheriting from `DeprecationWarning`, and then have a single `warnings.simplefilter("default", KedroDeprecationWarning)` in our `kedro/__init__.py`.
- Or, if we are to create a custom class, maybe we can just make it a child of `FutureWarning`. Would be weird to use `Deprecation` in a `Future` child, but maybe incurring in that internal inconsistency will minimize confusion while still making the intent clear.

## Possible Alternatives
Alternatively, we could wrap our filter as advised by the CPython docs:

```python
import sys

if not sys.warnoptions:
    import warnings
    warnings.simplefilter(...)
```

However, this would probably not solve the core issue.


Discussion:

Resolving patch:
diff --git a/kedro/io/__init__.py b/kedro/io/__init__.py
--- a/kedro/io/__init__.py
+++ b/kedro/io/__init__.py
@@ -3,9 +3,7 @@
 """
 from __future__ import annotations
 
-import warnings
-
-from .cached_dataset import CachedDataset
+from .cached_dataset import CachedDataset, CachedDataSet
 from .core import (
     AbstractDataSet,
     AbstractVersionedDataSet,
@@ -15,17 +13,14 @@
     Version,
 )
 from .data_catalog import DataCatalog
-from .lambda_dataset import LambdaDataset
-from .memory_dataset import MemoryDataset
-from .partitioned_dataset import IncrementalDataset, PartitionedDataset
-
-with warnings.catch_warnings():
-    warnings.simplefilter("ignore", DeprecationWarning)
-    from .cached_dataset import CachedDataSet
-    from .lambda_dataset import LambdaDataSet
-    from .memory_dataset import MemoryDataSet
-    from .partitioned_dataset import IncrementalDataSet, PartitionedDataSet
-
+from .lambda_dataset import LambdaDataset, LambdaDataSet
+from .memory_dataset import MemoryDataset, MemoryDataSet
+from .partitioned_dataset import (
+    IncrementalDataset,
+    IncrementalDataSet,
+    PartitionedDataset,
+    PartitionedDataSet,
+)
 
 # https://github.com/pylint-dev/pylint/issues/4300#issuecomment-1043601901
 DataSetError: type[Exception]
diff --git a/kedro/io/core.py b/kedro/io/core.py
--- a/kedro/io/core.py
+++ b/kedro/io/core.py
@@ -22,8 +22,6 @@
 
 from kedro.utils import load_obj
 
-warnings.simplefilter("default", DeprecationWarning)
-
 VERSION_FORMAT = "%Y-%m-%dT%H.%M.%S.%fZ"
 VERSIONED_FLAG_KEY = "versioned"
 VERSION_KEY = "version"


Test patch: