Repository: kedro-org/kedro

Issue:
Improve Jupyter workflow when configuration is broken
## Description
> Is your feature request related to a problem? A clear and concise description of what the problem is: "I'm always frustrated when ..."

Using the spaceflights demo on 0.18.1 if I open a `kedro jupyter notebook` session and write `catalog.list()` I will get the following:

![image](https://user-images.githubusercontent.com/35801847/172139364-e37f8a47-69cf-4749-9ab4-3a9e55ede72e.png)

If I break the `catalog.yml` file so that it is unparsable and run `kedro catalog list` (or any other command like `kedro run` I will get a helpful parse error in the terminal: 

<img width="654" alt="image" src="https://user-images.githubusercontent.com/35801847/172139490-a079095e-52d7-4c80-a8d3-15c5a675239e.png">

However, if I start a `kedro jupyter notebook` session in this state, I get no warning or messaging that Kedro is unable to create the `catalog`, `session` and other global variables. As a user I get this confusing state where the `catalog` is no longer available to me:

<img width="636" alt="image" src="https://user-images.githubusercontent.com/35801847/172139614-8dabbc13-38a4-4785-9afe-bbddf77b974a.png">


## Context
> Why is this change important to you? How would you use it? How can it benefit other users?

I think novice users can get stuck here with no direction as to why the catalog is not available.

## Possible Implementation

Surface warning to the jupyter context, perhaps even make a dummy `catalog`, `context`, `session` objects available that expose the log message when called...


Discussion:
I like this suggestion 👍  Does the error message appear in the console where you run `kedro jupyter notebook` or nowhere at all?

We used to have some sort of `startup_error` variable, not sure what happened to it. We actually need to improve error handling in `load_ipython_extension` in general anyway, so it probably makes sense to consider this issue at the same time as that so we have a solution that makes sense. I'll try to figure out the best way to approach these would be.
I don't think so - this is what comes up in the 2nd run:


```
kedro jupyter notebook
2022-06-06 10:55:55,418 - kedro.framework.cli.hooks.manager - INFO - Registered CLI hooks from 1 installed plugin(s): kedro-telemetry-0.2.1
Jupyter kernel kedro_jupytertest already exists and will be used.
/Users/.../miniconda3/envs/kedro-temp/bin/python -m jupyter notebook --MultiKernelManager.default_kernel_name=kedro_jupytertest
[I 2022-06-06 10:55:59.524 LabApp] JupyterLab extension loaded from /Users/.../miniconda3/envs/kedro-temp/lib/python3.8/site-packages/jupyterlab
[I 2022-06-06 10:55:59.524 LabApp] JupyterLab application directory is /Users/.../miniconda3/envs/kedro-temp/share/jupyter/lab
[I 10:55:59.530 NotebookApp] Serving notebooks from local directory: /Users/.../Documents/projects/jupytertest
[I 10:55:59.530 NotebookApp] Jupyter Notebook 6.4.11 is running at:
[I 10:55:59.530 NotebookApp] http://localhost:8888/?token=7e16104fba399d8efa9d8700ad3e5cb9a0ce9b9bfd5e9fd1
[I 10:55:59.530 NotebookApp]  or http://127.0.0.1:8888/?token=7e16104fba399d8efa9d8700ad3e5cb9a0ce9b9bfd5e9fd1
[I 10:55:59.530 NotebookApp] Use Control-C to stop this server and shut down all kernels (twice to skip confirmation).
[C 10:55:59.537 NotebookApp]

    To access the notebook, open this file in a browser:
        file:///Users/.../Library/Jupyter/runtime/nbserver-5978-open.html
    Or copy and paste one of these URLs:
        http://localhost:8888/?token=7e16104fba399d8efa9d8700ad3e5cb9a0ce9b9bfd5e9fd1
     or http://127.0.0.1:8888/?token=7e16104fba399d8efa9d8700ad3e5cb9a0ce9b9bfd5e9fd1
[W 10:56:01.358 NotebookApp] Kernel Provisioning: The 'local-provisioner' is not found.  This is likely due to the presence of multiple jupyter_client distributions and a previous distribution is being used as the source for entrypoints - which does not include 'local-provisioner'.  That distribution should be removed such that only the version-appropriate distribution remains (version >= 7).  Until then, a 'local-provisioner' entrypoint will be automatically constructed and used.
    The candidate distribution locations are: ['/Users/.../miniconda3/envs/kedro-temp/lib/python3.8/site-packages/jupyter_client-7.3.2.dist-info']
[I 10:56:05.564 NotebookApp] Kernel started: a2c15a7b-22ff-400d-85be-48d61a39bb74, name: kedro_jupytertest
[IPKernelApp] ERROR | No such comm target registered: jupyter.widget.control
[IPKernelApp] WARNING | No such comm: 18ba389e-4b85-471c-a2c1-c8e83a925b94
[W 10:56:18.898 NotebookApp] 404 GET /api/kernels/b3ffafb1-e8b3-404f-b08b-722f43d6bfcd/channels?session_id=4051ac1d7f0248a0b336e473cc59b596 (::1): Kernel does not exist: b3ffafb1-e8b3-404f-b08b-722f43d6bfcd
[W 10:56:18.909 NotebookApp] 404 GET /api/kernels/b3ffafb1-e8b3-404f-b08b-722f43d6bfcd/channels?session_id=4051ac1d7f0248a0b336e473cc59b596 (::1) 11.950000ms referer=None
[I 10:56:46.782 NotebookApp] Saving file at /notebooks/Untitled.ipynb
[W 10:57:23.951 NotebookApp] 404 GET /api/kernels/b3ffafb1-e8b3-404f-b08b-722f43d6bfcd/channels?session_id=4051ac1d7f0248a0b336e473cc59b596 (::1): Kernel does not exist: b3ffafb1-e8b3-404f-b08b-722f43d6bfcd
[W 10:57:23.952 NotebookApp] 404 GET /api/kernels/b3ffafb1-e8b3-404f-b08b-722f43d6bfcd/channels?session_id=4051ac1d7f0248a0b336e473cc59b596 (::1) 1.990000ms referer=None
[I 11:01:43.490 NotebookApp] Starting buffering for a2c15a7b-22ff-400d-85be-48d61a39bb74:8b65c945c8f74bd1a009a0cca76f1844
```

Resolving patch:
diff --git a/kedro/extras/extensions/ipython.py b/kedro/extras/extensions/ipython.py
--- a/kedro/extras/extensions/ipython.py
+++ b/kedro/extras/extensions/ipython.py
@@ -1,4 +1,4 @@
-# pylint: disable=import-outside-toplevel,global-statement,invalid-name
+# pylint: disable=import-outside-toplevel,global-statement,invalid-name,too-many-locals
 """
 This script creates an IPython extension to load Kedro-related variables in
 local scope.
@@ -8,9 +8,6 @@
 from pathlib import Path
 from typing import Any, Dict
 
-from IPython import get_ipython
-from IPython.core.magic import needs_local_scope, register_line_magic
-
 logger = logging.getLogger(__name__)
 default_project_path = Path.cwd()
 
@@ -39,9 +36,10 @@ def reload_kedro(
 ):
     """Line magic which reloads all Kedro default variables.
     Setting the path will also make it default for subsequent calls.
-
-
     """
+    from IPython import get_ipython
+    from IPython.core.magic import needs_local_scope, register_line_magic
+
     from kedro.framework.cli import load_entry_points
     from kedro.framework.project import LOGGING  # noqa # pylint:disable=unused-import
     from kedro.framework.project import configure_project, pipelines
@@ -63,7 +61,6 @@ def reload_kedro(
     session = KedroSession.create(
         metadata.package_name, default_project_path, env=env, extra_params=extra_params
     )
-    logger.debug("Loading the context from %s", default_project_path)
     context = session.load_context()
     catalog = context.catalog
 
@@ -95,12 +92,11 @@ def load_ipython_extension(ipython):
 
     default_project_path = _find_kedro_project(Path.cwd())
 
-    try:
-        reload_kedro(default_project_path)
-    except (ImportError, ModuleNotFoundError):
-        logger.error("Kedro appears not to be installed in your current environment.")
-    except Exception:  # pylint: disable=broad-except
+    if default_project_path is None:
         logger.warning(
             "Kedro extension was registered but couldn't find a Kedro project. "
             "Make sure you run '%reload_kedro <project_root>'."
         )
+        return
+
+    reload_kedro(default_project_path)


Test patch:
diff --git a/tests/extras/extensions/test_ipython.py b/tests/extras/extensions/test_ipython.py
--- a/tests/extras/extensions/test_ipython.py
+++ b/tests/extras/extensions/test_ipython.py
@@ -13,12 +13,7 @@ def project_path(mocker, tmp_path):
 
 
 @pytest.fixture(autouse=True)
-def cleanup_session():
-    yield
-
-
-@pytest.fixture()
-def pipeline_cleanup():
+def cleanup_pipeline():
     yield
     from kedro.framework.project import pipelines
 
@@ -31,7 +26,6 @@ def pipeline_cleanup():
 
 
 class TestLoadKedroObjects:
-    @pytest.mark.usefixtures("pipeline_cleanup")
     def test_load_kedro_objects(
         self, tmp_path, mocker, caplog
     ):  # pylint: disable=too-many-locals
@@ -62,18 +56,18 @@ def my_register_pipeline():
         mocker.patch(
             "kedro.framework.startup.bootstrap_project", return_value=fake_metadata
         )
-        mock_line_magic = mocker.MagicMock()
+        mock_line_magic = mocker.Mock()
         mock_line_magic.__name__ = "abc"
         mocker.patch(
             "kedro.framework.cli.load_entry_points", return_value=[mock_line_magic]
         )
         mock_register_line_magic = mocker.patch(
-            "kedro.extras.extensions.ipython.register_line_magic"
+            "IPython.core.magic.register_line_magic"
         )
         mock_session_create = mocker.patch(
             "kedro.framework.session.KedroSession.create"
         )
-        mock_ipython = mocker.patch("kedro.extras.extensions.ipython.get_ipython")
+        mock_ipython = mocker.patch("IPython.get_ipython")
 
         reload_kedro(kedro_path)
 
@@ -113,18 +107,18 @@ def test_load_kedro_objects_extra_args(self, tmp_path, mocker):
         mocker.patch(
             "kedro.framework.startup.bootstrap_project", return_value=fake_metadata
         )
-        mock_line_magic = mocker.MagicMock()
+        mock_line_magic = mocker.Mock()
         mock_line_magic.__name__ = "abc"
         mocker.patch(
             "kedro.framework.cli.load_entry_points", return_value=[mock_line_magic]
         )
         mock_register_line_magic = mocker.patch(
-            "kedro.extras.extensions.ipython.register_line_magic"
+            "IPython.core.magic.register_line_magic"
         )
         mock_session_create = mocker.patch(
             "kedro.framework.session.KedroSession.create"
         )
-        mock_ipython = mocker.patch("kedro.extras.extensions.ipython.get_ipython")
+        mock_ipython = mocker.patch("IPython.get_ipython")
 
         reload_kedro(tmp_path, env="env1", extra_params={"key": "val"})
 
@@ -141,18 +135,6 @@ def test_load_kedro_objects_extra_args(self, tmp_path, mocker):
         )
         assert mock_register_line_magic.call_count == 1
 
-    def test_load_kedro_objects_not_in_kedro_project(self, tmp_path, mocker):
-        mocker.patch(
-            "kedro.framework.startup._get_project_metadata", side_effect=RuntimeError
-        )
-        mock_ipython = mocker.patch("kedro.extras.extensions.ipython.get_ipython")
-
-        with pytest.raises(RuntimeError):
-            reload_kedro(tmp_path)
-        assert not mock_ipython().called
-        assert not mock_ipython().push.called
-
-    @pytest.mark.usefixtures("pipeline_cleanup")
     def test_load_kedro_objects_no_path(self, tmp_path, caplog, mocker):
         from kedro.extras.extensions.ipython import default_project_path
 
@@ -181,14 +163,14 @@ def my_register_pipeline():
         mocker.patch(
             "kedro.framework.startup.bootstrap_project", return_value=fake_metadata
         )
-        mock_line_magic = mocker.MagicMock()
+        mock_line_magic = mocker.Mock()
         mock_line_magic.__name__ = "abc"
         mocker.patch(
             "kedro.framework.cli.load_entry_points", return_value=[mock_line_magic]
         )
-        mocker.patch("kedro.extras.extensions.ipython.register_line_magic")
+        mocker.patch("IPython.core.magic.register_line_magic")
         mocker.patch("kedro.framework.session.KedroSession.load_context")
-        mocker.patch("kedro.extras.extensions.ipython.get_ipython")
+        mocker.patch("IPython.get_ipython")
 
         reload_kedro()
 
@@ -203,36 +185,36 @@ def my_register_pipeline():
 
 
 class TestLoadIPythonExtension:
-    @pytest.mark.parametrize(
-        "error,expected_log_message,level",
-        [
-            (
-                ImportError,
-                "Kedro appears not to be installed in your current environment.",
-                "ERROR",
-            ),
-            (
-                RuntimeError,
-                "Kedro extension was registered but couldn't find a Kedro project. "
-                "Make sure you run '%reload_kedro <project_root>'.",
-                "WARNING",
-            ),
-        ],
-    )
-    def test_load_extension_not_in_kedro_env_or_project(
-        self, error, expected_log_message, level, mocker, caplog
-    ):
-        mocker.patch("kedro.framework.startup._get_project_metadata", side_effect=error)
-        mock_ipython = mocker.patch("kedro.extras.extensions.ipython.get_ipython")
-
-        load_ipython_extension(mocker.MagicMock())
+    def test_load_extension_missing_dependency(self, mocker):
+        mocker.patch(
+            "kedro.extras.extensions.ipython.reload_kedro", side_effect=ImportError
+        )
+        mocker.patch(
+            "kedro.extras.extensions.ipython._find_kedro_project",
+            return_value=mocker.Mock(),
+        )
+        mock_ipython = mocker.patch("IPython.get_ipython")
+
+        with pytest.raises(ImportError):
+            load_ipython_extension(mocker.Mock())
+
+        assert not mock_ipython().called
+        assert not mock_ipython().push.called
+
+    def test_load_extension_not_in_kedro_project(self, mocker, caplog):
+        mocker.patch(
+            "kedro.extras.extensions.ipython._find_kedro_project", return_value=None
+        )
+        mock_ipython = mocker.patch("IPython.get_ipython")
+
+        load_ipython_extension(mocker.Mock())
 
         assert not mock_ipython().called
         assert not mock_ipython().push.called
 
-        log_messages = [
-            record.getMessage()
-            for record in caplog.records
-            if record.levelname == level
-        ]
-        assert log_messages == [expected_log_message]
+        log_messages = [record.getMessage() for record in caplog.records]
+        expected_message = (
+            "Kedro extension was registered but couldn't find a Kedro project. "
+            "Make sure you run '%reload_kedro <project_root>'."
+        )
+        assert expected_message in log_messages