Repository: kedro-org/kedro

Issue:
Improve error messages for modular pipelines
## Description
Some errors that happen when using modular pipelines could be more helpful.

For example: if a node has a non-existing dataset as input, the `ModularPipelineError` will actually include the name of the existing dataset in the catalog. You can reproduce that by taking the code of our tutorial:

https://github.com/kedro-org/kedro/blob/f8230cdbc653f4c66194c34b91fd74b919ae7183/docs/source/tutorial/add_another_pipeline.md?plain=1#L149

And changing one of the inputs to `model_input_table_NOT_FOUND` will raise this error:

```
ModularPipelineError: Failed to map datasets and/or parameters: model_input_table
```

(source: https://www.linen.dev/s/kedro/t/12314014/hi-everyone-i-am-trying-to-use-the-modular-pipeline-module-b#27abe516-0718-43d4-8924-2bd965f64d22)

Another one that recently confused an internal user: the "Inputs should be free inputs to the pipeline". A free input is "not an output from another node, thus unbound or free" (@idanov).

https://github.com/kedro-org/kedro/blob/f8230cdbc653f4c66194c34b91fd74b919ae7183/kedro/pipeline/modular_pipeline.py#L54C1-L55

## Possible Implementation
In the first case, maybe the error checking code should first check the nodes inputs, to give a more helpful error message.

In the second case, the text could say for example "Inputs must not be outputs from another node".

## Possible Alternatives
(Optional) Describe any alternative solutions or features you've considered.



Discussion:
@astrojuanlu Can you add some steps to reproduce these confusing error message(s)?
I haven't worked on a reproducer yet but here's another user puzzled by the error message https://www.linen.dev/s/kedro/t/13226590/hi-everyone-i-m-having-a-bit-of-hard-time-understanding-what#055add79-fc39-450f-98eb-b8a8746cd2e7

> Could some kindly unpack / explain it ?
So I have lived experience teaching people to use the feature and this not being straightforward - a fuzzy suggestion workflow would go a long way I think.
So, the way to reproduce this error is, starting from the spaceflights tutorial, to create a `data_science/pipeline.py` as follows:

```python
from kedro.pipeline import Pipeline, node, pipeline

from .nodes import evaluate_model, split_data, train_model


def create_pipeline(**kwargs) -> Pipeline:
    pipeline_instance = pipeline(
        [
            node(
                func=split_data,
                # Note wrong input name
                inputs=["model_input_table_NOT_FOUND", "params:model_options"],
                outputs=["X_train", "X_test", "y_train", "y_test"],
                name="split_data_node",
            ),
            node(
                func=train_model,
                inputs=["X_train", "y_train"],
                outputs="regressor",
                name="train_model_node",
            ),
            node(
                func=evaluate_model,
                inputs=["regressor", "X_test", "y_test"],
                outputs=None,
                name="evaluate_model_node",
            ),
        ]
    )

    ds_pipeline_1 = pipeline(
        pipe=pipeline_instance,
        inputs="model_input_table",
        namespace="active_modelling_pipeline",
    )
    ds_pipeline_2 = pipeline(
        pipe=pipeline_instance,
        inputs="model_input_table",
        namespace="candidate_modelling_pipeline",
    )

    return ds_pipeline_1 + ds_pipeline_2
```

And then the error will be

```
ModularPipelineError: Failed to map datasets and/or parameters: model_input_table
```

Why is this confusing? Because `model_input_table` is a well-defined dataset in the catalog. But the error actually means that there's a mismatch between the input declared in the namepaced pipeline and the one originally declared in the pipeline instance.
I think it would be very helpful to suggest:

- partial matches which helps solves the namespace mismatch issue
- fuzzy matches for typos 



Patch:
diff --git a/kedro/pipeline/modular_pipeline.py b/kedro/pipeline/modular_pipeline.py
index 6e89b810b4..9ac2d6a1d1 100644
--- a/kedro/pipeline/modular_pipeline.py
+++ b/kedro/pipeline/modular_pipeline.py
@@ -2,6 +2,7 @@
 from __future__ import annotations
 
 import copy
+import difflib
 from typing import AbstractSet, Iterable
 
 from kedro.pipeline.node import Node
@@ -52,10 +53,14 @@ def _validate_inputs_outputs(
     free_inputs = {_strip_transcoding(i) for i in pipe.inputs()}
 
     if not inputs <= free_inputs:
-        raise ModularPipelineError("Inputs should be free inputs to the pipeline")
+        raise ModularPipelineError(
+            "Inputs must not be outputs from another node in the same pipeline"
+        )
 
     if outputs & free_inputs:
-        raise ModularPipelineError("Outputs can't contain free inputs to the pipeline")
+        raise ModularPipelineError(
+            "All outputs must be generated by some node within the pipeline"
+        )
 
 
 def _validate_datasets_exist(
@@ -64,16 +69,25 @@ def _validate_datasets_exist(
     parameters: AbstractSet[str],
     pipe: Pipeline,
 ) -> None:
+    """Validate that inputs, parameters and outputs map correctly onto the provided nodes."""
     inputs = {_strip_transcoding(k) for k in inputs}
     outputs = {_strip_transcoding(k) for k in outputs}
 
     existing = {_strip_transcoding(ds) for ds in pipe.datasets()}
     non_existent = (inputs | outputs | parameters) - existing
     if non_existent:
-        raise ModularPipelineError(
-            f"Failed to map datasets and/or parameters: "
-            f"{', '.join(sorted(non_existent))}"
+        sorted_non_existent = sorted(non_existent)
+        possible_matches = []
+        for non_existent_input in sorted_non_existent:
+            possible_matches += difflib.get_close_matches(non_existent_input, existing)
+
+        error_msg = f"Failed to map datasets and/or parameters onto the nodes provided: {', '.join(sorted_non_existent)}"
+        suggestions = (
+            f" - did you mean one of these instead: {', '.join(possible_matches)}"
+            if possible_matches
+            else ""
         )
+        raise ModularPipelineError(error_msg + suggestions)
 
 
 def _get_dataset_names_mapping(


Test patch:
diff --git a/tests/pipeline/test_modular_pipeline.py b/tests/pipeline/test_modular_pipeline.py
index bfb2e347c3..c1e76867b5 100644
--- a/tests/pipeline/test_modular_pipeline.py
+++ b/tests/pipeline/test_modular_pipeline.py
@@ -217,16 +217,67 @@ def test_empty_output(self):
             ),
         ],
     )
-    def test_missing_dataset_name(
+    def test_missing_dataset_name_no_suggestion(
         self, func, inputs, outputs, inputs_map, outputs_map, expected_missing
     ):  # noqa: PLR0913
         raw_pipeline = modular_pipeline([node(func, inputs, outputs)])
 
-        with pytest.raises(ModularPipelineError, match=r"Failed to map datasets") as e:
+        with pytest.raises(
+            ModularPipelineError, match=r"Failed to map datasets and/or parameters"
+        ) as e:
             pipeline(
                 raw_pipeline, namespace="PREFIX", inputs=inputs_map, outputs=outputs_map
             )
         assert ", ".join(expected_missing) in str(e.value)
+        assert " - did you mean one of these instead: " not in str(e.value)
+
+    @pytest.mark.parametrize(
+        "func, inputs, outputs, inputs_map, outputs_map, expected_missing, expected_suggestion",
+        [
+            # Testing inputs
+            (identity, "IN", "OUT", {"I": "in"}, {}, ["I"], ["IN"]),
+            (
+                biconcat,
+                ["IN1", "IN2"],
+                "OUT",
+                {"IN": "input"},
+                {},
+                ["IN"],
+                ["IN2", "IN1"],
+            ),
+            # Testing outputs
+            (identity, "IN", "OUT", {}, {"OUT_": "output"}, ["OUT_"], ["OUT"]),
+            (
+                identity,
+                "IN",
+                ["OUT1", "OUT2"],
+                {},
+                {"OUT": "out"},
+                ["OUT"],
+                ["OUT2", "OUT1"],
+            ),
+        ],
+    )
+    def test_missing_dataset_with_suggestion(
+        self,
+        func,
+        inputs,
+        outputs,
+        inputs_map,
+        outputs_map,
+        expected_missing,
+        expected_suggestion,
+    ):
+        raw_pipeline = modular_pipeline([node(func, inputs, outputs)])
+
+        with pytest.raises(
+            ModularPipelineError, match=r"Failed to map datasets and/or parameters"
+        ) as e:
+            pipeline(
+                raw_pipeline, namespace="PREFIX", inputs=inputs_map, outputs=outputs_map
+            )
+        assert ", ".join(expected_missing) in str(e.value)
+        assert ", ".join(expected_suggestion) in str(e.value)
 
     def test_node_properties_preserved(self):
         """
@@ -378,11 +429,11 @@ def test_non_existent_parameters_mapped(self):
             ]
         )
 
-        pattern = r"Failed to map datasets and/or parameters: params:beta"
+        pattern = r"Failed to map datasets and/or parameters onto the nodes provided: params:beta"
         with pytest.raises(ModularPipelineError, match=pattern):
             pipeline(raw_pipeline, parameters={"beta": "gamma"})
 
-        pattern = r"Failed to map datasets and/or parameters: parameters"
+        pattern = r"Failed to map datasets and/or parameters onto the nodes provided: parameters"
         with pytest.raises(ModularPipelineError, match=pattern):
             pipeline(raw_pipeline, parameters={"parameters": "some_yaml_dataset"})
 
@@ -394,7 +445,7 @@ def test_bad_inputs_mapping(self):
             ]
         )
 
-        pattern = "Inputs should be free inputs to the pipeline"
+        pattern = "Inputs must not be outputs from another node in the same pipeline"
         with pytest.raises(ModularPipelineError, match=pattern):
             pipeline(raw_pipeline, inputs={"AA": "CC"})
 
@@ -406,7 +457,7 @@ def test_bad_outputs_mapping(self):
             ]
         )
 
-        pattern = "Outputs can't contain free inputs to the pipeline"
+        pattern = "All outputs must be generated by some node within the pipeline"
         with pytest.raises(ModularPipelineError, match=pattern):
             pipeline(raw_pipeline, outputs={"A": "C"})