Repository: RasaHQ/rasa
Pull request: warn for invalid keys for components
**Proposed changes**:
- Warn when a user provides a key for a component that is not part of the component's allowed keys
- closes #6966

**Status (please check what you already did)**:
- [x] added some tests for the functionality
- [ ] updated the documentation
- [x] updated the changelog (please check [changelog](https://github.com/RasaHQ/rasa/tree/master/changelog) for instructions)
- [x] reformat files using `black` (please check [Readme](https://github.com/RasaHQ/rasa#code-style) for instructions)

Should this raise `InvalidConfigError` instead, the way `validate_requirements` does?


Problem statement:
Warn for invalid keys for classifiers

**Rasa version**: 2.0.0

**Issue**:

If you pass an invalid key to a classifier no warning or error is thrown during training

**Command or request that led to lack of error**:
```
rasa train
```

**Content of configuration file (config.yml)** (if relevant):
```yml
pipeline
- name: FallbackClassifier
   confidence_threshold: 0.7
```

(should be `threshold`)

Hints from issue discussion:
Here's a similar issue with responses. `button_type` is not valid at this location, but `validate` and `train` don't throw an error.

```
  utter_ask_feedback:
  - buttons:
    - button_type: inline
    - payload: /feedback{"feedback_value":"positive"}
      title: 👍
    - payload: /feedback{"feedback_value":"negative"}
      title: 👎
    text: How is this conversation going?
```
The trouble with responses is depending on the channel, those keys may be valid or not. With built in classifiers, the possible keys are known.
would be great to find a general solution which also works for other NLU components than classifiers
@wochinge one way would be if each component class had an `options` or `settings` property that was a list of legitimate keys for that component. Then it could be validated  like the `name` property is validated. wdyt?
The [defaults](https://github.com/RasaHQ/rasa/blob/8313c8739fe6aeb3666cf6b145c0af583662fa0d/rasa/nlu/classifiers/fallback_classifier.py#L28) should already give us that information 🤔 
Do all possible options have defaults? 
looks like it. I'll use that then
wait - so you're working on this?
I'm about to unless someone else started already?
Fine for me - it's just for the 2.1 release plan 🚀

Commit message: remove blank lines after docstrings

Patch to review:
diff --git a/changelog/6966.improvement.md b/changelog/6966.improvement.md
new file mode 100644
index 000000000000..fb0ebd4b58c0
--- /dev/null
+++ b/changelog/6966.improvement.md
@@ -0,0 +1 @@
+Added warning for when an option is provided for a [component](https://rasa.com/docs/rasa/components) that is not listed as a key in the defaults for that component.
diff --git a/rasa/nlu/components.py b/rasa/nlu/components.py
index 8d27f91d8356..6c267053dd2c 100644
--- a/rasa/nlu/components.py
+++ b/rasa/nlu/components.py
@@ -91,13 +91,40 @@ def validate_requirements(component_names: List[Optional[Text]]) -> None:
         )
 
 
+def validate_component_keys(
+    component: "Component", component_config: Dict[Text, Any]
+) -> None:
+    """Validates that all keys for a component are valid.
+
+    Raises:
+        UserWarning: If the component has a key specified that is not used
+            by the component class, it is likely a mistake in the pipeline
+
+    Args:
+        component: The component class
+
+        component_config: The user-provided config for the component in the pipeline
+    """
+    component_name = component_config.get("name")
+    allowed_keys = set(component.defaults.keys())
+    provided_keys = set(component_config.keys())
+    provided_keys.remove("name")
+    list_separator = "\n- "
+    for key in provided_keys:
+        if key not in allowed_keys:
+            rasa.shared.utils.io.raise_warning(
+                f"You have provided an invalid key `{key}` for component `{component_name}` in your pipeline. "
+                f"Valid options for `{component_name}` are:\n- "
+                f"{list_separator.join(allowed_keys)}"
+            )
+
+
 def validate_empty_pipeline(pipeline: List["Component"]) -> None:
     """Ensures the pipeline is not empty.
 
     Args:
         pipeline: the list of the :class:`rasa.nlu.components.Component`.
     """
-
     if len(pipeline) == 0:
         raise InvalidConfigError(
             "Can not train an empty pipeline. "
diff --git a/rasa/nlu/model.py b/rasa/nlu/model.py
index bc89023c9a7e..1d52e5d198ac 100644
--- a/rasa/nlu/model.py
+++ b/rasa/nlu/model.py
@@ -161,13 +161,13 @@ def _build_pipeline(
         self, cfg: RasaNLUModelConfig, component_builder: ComponentBuilder
     ) -> List[Component]:
         """Transform the passed names of the pipeline components into classes."""
-
         pipeline = []
 
         # Transform the passed names of the pipeline components into classes
         for i in range(len(cfg.pipeline)):
             component_cfg = cfg.for_component(i)
             component = component_builder.create_component(component_cfg, cfg)
+            components.validate_component_keys(component, cfg.pipeline[i])
             pipeline.append(component)
 
         if not self.skip_validation:
diff --git a/tests/nlu/test_components.py b/tests/nlu/test_components.py
index 3362b8848d76..6717a7eabd37 100644
--- a/tests/nlu/test_components.py
+++ b/tests/nlu/test_components.py
@@ -200,3 +200,17 @@ async def test_validate_requirements_raises_exception_on_component_without_name(
         await train(
             _config, data="./data/examples/rasa/demo-rasa.json", path=str(tmp_path),
         )
+
+
+async def test_validate_component_keys_raises_warning_on_invalid_key(tmp_path: Path,):
+    _config = RasaNLUModelConfig(
+        # config with a component that does not have a `confidence_threshold ` property
+        {"pipeline": [{"name": "WhitespaceTokenizer", "confidence_threshold": 0.7}]}
+    )
+
+    with pytest.warns(UserWarning) as record:
+        await train(
+            _config, data="./data/examples/rasa/demo-rasa.json", path=str(tmp_path),
+        )
+
+    assert "You have provided an invalid key" in record[0].message.args[0]


Code review comments:
diff_hunk: @@ -161,13 +161,13 @@ def _build_pipeline(
         self, cfg: RasaNLUModelConfig, component_builder: ComponentBuilder
     ) -> List[Component]:
         """Transform the passed names of the pipeline components into classes."""
-
         pipeline = []
 
         # Transform the passed names of the pipeline components into classes
         for i in range(len(cfg.pipeline)):
path: rasa/nlu/model.py
text: @user1:
what do you think?
```suggestion
        for index, pipeline_component in enumerate(cfg.pipeline):
```

@author:
that is more readable

diff_hunk: @@ -200,3 +200,17 @@ async def test_validate_requirements_raises_exception_on_component_without_name(
         await train(
             _config, data="./data/examples/rasa/demo-rasa.json", path=str(tmp_path),
         )
+
+
+async def test_validate_component_keys_raises_warning_on_invalid_key(tmp_path: Path,):
+    _config = RasaNLUModelConfig(
+        # config with a component that does not have a `confidence_threshold ` property
+        {"pipeline": [{"name": "WhitespaceTokenizer", "confidence_threshold": 0.7}]}
+    )
+
+    with pytest.warns(UserWarning) as record:
+        await train(
+            _config, data="./data/examples/rasa/demo-rasa.json", path=str(tmp_path),
path: tests/nlu/test_components.py
text: @user1:
```suggestion
            _config, data=DEFAULT_DATA_PATH, path=str(tmp_path),
```

diff_hunk: @@ -0,0 +1 @@
+Added warning for when an option is provided for a [component](https://rasa.com/docs/rasa/components) that is not listed as a key in the defaults for that component.
path: changelog/6966.improvement.md
text: @user1:
please use the relative path
```suggestion
Added warning for when an option is provided for a [component](components.mdx) that is not listed as a key in the defaults for that component.
```

diff_hunk: @@ -91,13 +91,40 @@ def validate_requirements(component_names: List[Optional[Text]]) -> None:
         )
 
 
+def validate_component_keys(
+    component: "Component", component_config: Dict[Text, Any]
+) -> None:
+    """Validates that all keys for a component are valid.
+
+    Raises:
+        UserWarning: If the component has a key specified that is not used
+            by the component class, it is likely a mistake in the pipeline
+
path: rasa/nlu/components.py
text: @user1:
[raises is for exceptions](https://google.github.io/styleguide/pyguide.html#doc-function-raises)
```suggestion
```

diff_hunk: @@ -91,13 +91,40 @@ def validate_requirements(component_names: List[Optional[Text]]) -> None:
         )
 
 
+def validate_component_keys(
+    component: "Component", component_config: Dict[Text, Any]
+) -> None:
+    """Validates that all keys for a component are valid.
+
+    Raises:
+        UserWarning: If the component has a key specified that is not used
+            by the component class, it is likely a mistake in the pipeline
+
+    Args:
+        component: The component class
+
path: rasa/nlu/components.py
text: @user1:
```suggestion
```

diff_hunk: @@ -91,13 +91,40 @@ def validate_requirements(component_names: List[Optional[Text]]) -> None:
         )
 
 
+def validate_component_keys(
+    component: "Component", component_config: Dict[Text, Any]
+) -> None:
+    """Validates that all keys for a component are valid.
+
+    Raises:
+        UserWarning: If the component has a key specified that is not used
+            by the component class, it is likely a mistake in the pipeline
+
+    Args:
+        component: The component class
+
+        component_config: The user-provided config for the component in the pipeline
+    """
+    component_name = component_config.get("name")
+    allowed_keys = set(component.defaults.keys())
+    provided_keys = set(component_config.keys())
+    provided_keys.remove("name")
path: rasa/nlu/components.py
text: @user1:
optional: `pop` with a default might be more robust in case the `name` item doesn't exist for some reason

@user1:
(`remove` raises in case the element is not in the set)

@author:
`pop` with a set is for arbitrary elements, could be a comprehension though
```suggestion
    provided_keys = (key for key in provided_keys if key != "name")
```