Repository: kedro-org/kedro

Issue:
Add `--conf-source` option to `kedro ipython`
## Description
Right now we can pass `env` and `params` as options with `kedro ipython` / `%reload_kedro`. Add `--conf-source` option as well.

## Context
User question from slack : 

> Speaking of  kedro ipython :
Is there a way to specify /override the  conf_source ? :slightly_smiling_face:

> My problem comes from the fact that we have a single pipeline / repo for all our clients.
To better enforce “tenant isolation”
I have organized conf and data as follow:
conf
└── client_A
    ├── base
    └── local
└── client_B
    ├── base
    └── local
data
├── client_A
│   └── 01_raw
│   └── 02_intermediate
│   └── ...
└── client_B
│   └── 01_raw
│   └── 02_intermediate
│   └── ...conf
I did so because I liked the idea of not being able to run the pipeline without being explicit about which client it should be done for…
(the structure above makes it so that kedro run without --conf-source will raise an error)
Another reason I did so (if I recall correctly) was to avoid having “duplicate keys” conflicts in the different yaml files across clients… 

## Possible Implementation
Changes to be made in `kedro/ipython/__init__.py`
PR for when `--conf-source` was added to `kedro run` - https://github.com/kedro-org/kedro/pull/2117



Discussion:
xref https://linen-slack.kedro.org/t/13906421/hi-again-speaking-of-kedro-ipython-is-there-a-way-to-specify#4b179fdf-0b5c-48cb-b287-6a53ed5b3bbb
Hi @ankatiyar, I would need some help to tackle this issue.

I read the PR https://github.com/kedro-org/kedro/pull/2117, but I am unsure how this should be implemented for `ipython`. 

Is it that it should be first added as an `@argument("--conf-source", type=str, default=None, help=CONF_SOURCE_HELP)` here?
https://github.com/kedro-org/kedro/blob/e1823eddaac80aa4af15764be46921b3b00355e2/kedro/ipython/__init__.py#L62


Then in function `def reload_kedro` add `conf-source: str = None` as an input
https://github.com/kedro-org/kedro/blob/e1823eddaac80aa4af15764be46921b3b00355e2/kedro/ipython/__init__.py#L81


And should be used as an input to load `kedroSession`
```
session = KedroSession.create(
        metadata.package_name, project_path, env=env, extra_params=extra_params, conf-source=conf-source
    ) 
```
 https://github.com/kedro-org/kedro/blob/e1823eddaac80aa4af15764be46921b3b00355e2/kedro/ipython/__init__.py#L93

Please let me know what your ideas are.. 💡 
@laizaparizotto I think what you've suggested looks about right, might be some more things are required. I haven't looked into the implementation myself but I can and in the meantime, if you can try this out too and open a PR(or a draft PR), we could collaborate on the implementation. 
Sure, @ankatiyar.

I will make the changes and open a PR soon. Thank you for the help

Resolving patch:
diff --git a/kedro/ipython/__init__.py b/kedro/ipython/__init__.py
--- a/kedro/ipython/__init__.py
+++ b/kedro/ipython/__init__.py
@@ -14,7 +14,7 @@
 from IPython.core.magic_arguments import argument, magic_arguments, parse_argstring
 
 from kedro.framework.cli import load_entry_points
-from kedro.framework.cli.project import PARAMS_ARG_HELP
+from kedro.framework.cli.project import CONF_SOURCE_HELP, PARAMS_ARG_HELP
 from kedro.framework.cli.utils import ENV_HELP, _split_params
 from kedro.framework.project import (
     LOGGING,  # noqa
@@ -65,14 +65,17 @@ def load_ipython_extension(ipython):
     default=None,
     help=PARAMS_ARG_HELP,
 )
-def magic_reload_kedro(line: str, local_ns: dict[str, Any] = None):
+@argument("--conf-source", type=str, default=None, help=CONF_SOURCE_HELP)
+def magic_reload_kedro(
+    line: str, local_ns: dict[str, Any] = None, conf_source: str = None
+):
     """
     The `%reload_kedro` IPython line magic.
     See https://kedro.readthedocs.io/en/stable/notebooks_and_ipython/kedro_and_notebooks.html#reload-kedro-line-magic # noqa: line-too-long
     for more.
     """
     args = parse_argstring(magic_reload_kedro, line)
-    reload_kedro(args.path, args.env, args.params, local_ns)
+    reload_kedro(args.path, args.env, args.params, local_ns, args.conf_source)
 
 
 def reload_kedro(
@@ -80,6 +83,7 @@ def reload_kedro(
     env: str = None,
     extra_params: dict[str, Any] = None,
     local_namespace: dict[str, Any] | None = None,
+    conf_source: str = None,
 ) -> None:  # pragma: no cover
     """Function that underlies the %reload_kedro Line magic. This should not be imported
     or run directly but instead invoked through %reload_kedro."""
@@ -91,7 +95,11 @@ def reload_kedro(
     configure_project(metadata.package_name)
 
     session = KedroSession.create(
-        metadata.package_name, project_path, env=env, extra_params=extra_params
+        metadata.package_name,
+        project_path,
+        env=env,
+        extra_params=extra_params,
+        conf_source=conf_source,
     )
     context = session.load_context()
     catalog = context.catalog


Test patch:
diff --git a/tests/ipython/test_ipython.py b/tests/ipython/test_ipython.py
--- a/tests/ipython/test_ipython.py
+++ b/tests/ipython/test_ipython.py
@@ -116,7 +116,11 @@ def my_register_pipeline():
         reload_kedro()
 
         mock_session_create.assert_called_once_with(
-            PACKAGE_NAME, None, env=None, extra_params=None
+            PACKAGE_NAME,
+            None,
+            env=None,
+            extra_params=None,
+            conf_source=None,
         )
         _, kwargs = ipython_spy.call_args_list[0]
         variables = kwargs["variables"]
@@ -143,14 +147,18 @@ def my_register_pipeline():
         ipython_spy = mocker.spy(ipython, "push")
         dummy_env = "env"
         dummy_dict = {"key": "value"}
+        dummy_conf_source = "conf/"
 
-        reload_kedro(fake_metadata.project_path, "env", {"key": "value"})
+        reload_kedro(
+            fake_metadata.project_path, "env", {"key": "value"}, conf_source="conf/"
+        )
 
         mock_session_create.assert_called_once_with(
             PACKAGE_NAME,
             fake_metadata.project_path,
             env=dummy_env,
             extra_params=dummy_dict,
+            conf_source=dummy_conf_source,
         )
         _, kwargs = ipython_spy.call_args_list[0]
         variables = kwargs["variables"]
@@ -223,6 +231,7 @@ def test_load_extension_register_line_magic(self, mocker, ipython):
             "--env=base",
             "-e base",
             ". --env=base --params=key:val",
+            "--conf-source=new_conf",
         ],
     )
     def test_line_magic_with_valid_arguments(self, mocker, args, ipython):