Repository: RasaHQ/rasa
Pull request: Remove unfeaturized messages for some NLU components before training and prediction
**Proposed changes**:
- Fixes a bug (#10504) which was caused by `DIETClassifier` (`ResponseSelector`, `SklearnIntentClassifier` and `CRFEntityExtractor` have the same issue) trying to process message which didn't have required features. Implements removing unfeaturized messages for the above-mentioned components before training and prediction.


**Status (please check what you already did)**:
- [x] added some tests for the functionality
- [ ] updated the documentation
- [x] updated the changelog (please check [changelog](https://github.com/RasaHQ/rasa/tree/main/changelog) for instructions)
- [x] reformat files using `black` (please check [Readme](https://github.com/RasaHQ/rasa#code-style) for instructions)


Problem statement:
Empty (or some spaces) input text causes Runtime error
### Rasa Open Source version

3.0.0

### Rasa SDK version

3.0.0

### Rasa X version

_No response_

### Python version

3.8

### What operating system are you using?

Linux

### What happened?

In NLU shell, when I input text="" or text="     "(some spaces), it gets error as follow (full details in 'Relevant log output') :
`
rasa.engine.exceptions.GraphComponentException: Error running graph component for node run_DIETClassifier4.
`

But in RASA-2.8.6, everything goes well and gets following response:
`
{
  "text": "",
  "intent": {
    "name": null,
    "confidence": 0.0
  },
  "entities": []
}
`

I suppose this is a new bug.

### Command / Request

_No response_

### Relevant log output

```shell
Traceback (most recent call last):
File "~/venv/lib/python3.8/site-packages/rasa/engine/graph.py", line 461, in __call__
output = self._fn(self._component, **run_kwargs)
File "~/venv/lib/python3.8/site-packages/rasa/nlu/classifiers/diet_classifier.py", line 1003, in process
out = self._predict(message)
File "~/venv/lib/python3.8/site-packages/rasa/nlu/classifiers/diet_classifier.py", line 932, in _predict
return self.model.run_inference(model_data)
File "~/venv/lib/python3.8/site-packages/rasa/utils/tensorflow/models.py", line 318, in run_inference
] = self._rasa_predict(batch_in)
File "~/venv/lib/python3.8/site-packages/rasa/utils/tensorflow/models.py", line 280, in _rasa_predict
outputs = tf_utils.sync_to_numpy_or_python_type(self.predict_step(batch_in))
File "~/venv/lib/python3.8/site-packages/rasa/utils/tensorflow/models.py", line 228, in predict_step
return self.batch_predict(batch_in)
File "~/venv/lib/python3.8/site-packages/rasa/nlu/classifiers/diet_classifier.py", line 1721, in batch_predict
tf_batch_data = self.batch_to_model_data_format(
File "~/venv/lib/python3.8/site-packages/rasa/utils/tensorflow/models.py", line 473, in batch_to_model_data_format
tensor, idx = RasaModel._convert_sparse_features(
File "~/venv/lib/python3.8/site-packages/rasa/utils/tensorflow/models.py", line 518, in _convert_sparse_features
shape = [batch[idx + 2][i] for i in range(number_of_dimensions - 1)] + [
File "~/venv/lib/python3.8/site-packages/rasa/utils/tensorflow/models.py", line 518, in <listcomp>
shape = [batch[idx + 2][i] for i in range(number_of_dimensions - 1)] + [
IndexError: index 1 is out of bounds for axis 0 with size 1

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "~/venv/bin/rasa", line 8, in <module>
sys.exit(main())
File "~/venv/lib/python3.8/site-packages/rasa/__main__.py", line 121, in main
cmdline_arguments.func(cmdline_arguments)
File "~/venv/lib/python3.8/site-packages/rasa/cli/shell.py", line 91, in shell_nlu
rasa.nlu.run.run_cmdline(model)
File "~/venv/lib/python3.8/site-packages/rasa/nlu/run.py", line 25, in run_cmdline
result = asyncio.run(agent.parse_message(message))
File "/usr/lib/python3.8/asyncio/runners.py", line 44, in run
return loop.run_until_complete(main)
File "uvloop/loop.pyx", line 1456, in uvloop.loop.Loop.run_until_complete
File "~/venv/lib/python3.8/site-packages/rasa/core/agent.py", line 419, in parse_message
return await self.processor.parse_message(message)
File "~/venv/lib/python3.8/site-packages/rasa/core/processor.py", line 635, in parse_message
parse_data = self._parse_message_with_graph(message, only_output_properties)
File "~/venv/lib/python3.8/site-packages/rasa/core/processor.py", line 659, in _parse_message_with_graph
results = self.graph_runner.run(
File "~/venv/lib/python3.8/site-packages/rasa/engine/runner/dask.py", line 106, in run
dask_result = dask.get(run_graph, run_targets)
File "~/venv/lib/python3.8/site-packages/dask/local.py", line 560, in get_sync
return get_async(
File "~/venv/lib/python3.8/site-packages/dask/local.py", line 503, in get_async
for key, res_info, failed in queue_get(queue).result():
File "/usr/lib/python3.8/concurrent/futures/_base.py", line 437, in result
return self.__get_result()
File "/usr/lib/python3.8/concurrent/futures/_base.py", line 389, in __get_result
raise self._exception
File "~/venv/lib/python3.8/site-packages/dask/local.py", line 545, in submit
fut.set_result(fn(*args, **kwargs))
File "~/venv/lib/python3.8/site-packages/dask/local.py", line 237, in batch_execute_tasks
return [execute_task(*a) for a in it]
File "~/venv/lib/python3.8/site-packages/dask/local.py", line 237, in <listcomp>
return [execute_task(*a) for a in it]
File "~/venv/lib/python3.8/site-packages/dask/local.py", line 228, in execute_task
result = pack_exception(e, dumps)
File "~/venv/lib/python3.8/site-packages/dask/local.py", line 223, in execute_task
result = _execute_task(task, data)
File "~/venv/lib/python3.8/site-packages/dask/core.py", line 121, in _execute_task
return func(*(_execute_task(a, cache) for a in args))
File "~/venv/lib/python3.8/site-packages/rasa/engine/graph.py", line 468, in __call__
raise GraphComponentException(
rasa.engine.exceptions.GraphComponentException: Error running graph component for node run_DIETClassifier4.
```

**Definition of done**
- [ ] Fix bug

Hints from issue discussion:
Thanks for raising this issue, @sebwarnke will get back to you about it soon✨

###### Please also check out the [docs](https://rasa.com/docs/) and the [forum](https://forum.rasa.com/) in case your issue was raised there too 🤗
@sara-tagger any progress??
Hi @whusunny,
I couldn't reproduce the error.
Here's what I did:
```
rasa init
rasa train
rasa shell nlu
```
and when I write an empty string or some spaces I get the expected output with no errors. 
here's my configuration:
```
Rasa Version      :         3.0.0
Minimum Compatible Version: 3.0.0
Rasa SDK Version  :         3.0.0
Python Version    :         3.8.12
Operating System  :         Linux-5.11.0-43-generic-x86_64-with-glibc2.17
```
Can you help me reproduce it? What should I do differently?
Hi, @jupyterjazz 
Thanks for your feedback.

Using rasa init project, it indeed runs well. When I change the pipline config as follow then training, the error comes back.
```yml
language: zh
pipeline:
  - name: "MitieNLP"
    model: "total_word_feature_extractor_zh.dat"
  - name: "JiebaTokenizer"
  - name: "RegexFeaturizer"
  - name: "MitieFeaturizer"
  - name: "DIETClassifier"
    epochs: 30
  - name: "RegexEntityExtractor"
  - name: "EntitySynonymMapper"
  - name: "FallbackClassifier"
    threshold: 0.2
    ambiguity_threshold: 0.05
```

Yes, I am using rasa for Chinese:)

Commit message: ignore processing empty strings for some components

Patch to review:
diff --git a/changelog/10504.bugfix.md b/changelog/10504.bugfix.md
new file mode 100644
index 000000000000..fee280a2e193
--- /dev/null
+++ b/changelog/10504.bugfix.md
@@ -0,0 +1 @@
+Fixes a bug which caused machine learning components like `DIETClassifier`, `ResponseSelector`, `CRFEntityExtractor` and `SklearnIntentClassifier` to throw an error when a message with an empty text string was passed to them for processing.
\ No newline at end of file
diff --git a/rasa/nlu/classifiers/diet_classifier.py b/rasa/nlu/classifiers/diet_classifier.py
index bf5016ecf2ca..9a3041f9ab2c 100644
--- a/rasa/nlu/classifiers/diet_classifier.py
+++ b/rasa/nlu/classifiers/diet_classifier.py
@@ -1000,7 +1000,7 @@ def _predict_entities(
     def process(self, messages: List[Message]) -> List[Message]:
         """Augments the message with intents, entities, and diagnostic data."""
         for message in messages:
-            out = self._predict(message)
+            out = self._predict(message) if message.get(TEXT) else None
 
             if self.component_config[INTENT_CLASSIFICATION]:
                 label, label_ranking = self._predict_label(out)
diff --git a/rasa/nlu/classifiers/sklearn_intent_classifier.py b/rasa/nlu/classifiers/sklearn_intent_classifier.py
index e0a1516a5f06..0972617d4bc2 100644
--- a/rasa/nlu/classifiers/sklearn_intent_classifier.py
+++ b/rasa/nlu/classifiers/sklearn_intent_classifier.py
@@ -190,9 +190,9 @@ def _create_classifier(
     def process(self, messages: List[Message]) -> List[Message]:
         """Return the most likely intent and its probability for a message."""
         for message in messages:
-            if not self.clf:
-                # component is either not trained or didn't
-                # receive enough training data
+            if not self.clf or not message.get(TEXT):
+                # component is either not trained, didn't receive
+                # enough training data or received an empty string
                 intent = None
                 intent_ranking = []
             else:
diff --git a/rasa/nlu/extractors/crf_entity_extractor.py b/rasa/nlu/extractors/crf_entity_extractor.py
index e5f12d6d6155..0121428c1394 100644
--- a/rasa/nlu/extractors/crf_entity_extractor.py
+++ b/rasa/nlu/extractors/crf_entity_extractor.py
@@ -278,7 +278,7 @@ def process(self, messages: List[Message]) -> List[Message]:
 
     def extract_entities(self, message: Message) -> List[Dict[Text, Any]]:
         """Extract entities from the given message using the trained model(s)."""
-        if self.entity_taggers is None:
+        if self.entity_taggers is None or not message.get(TEXT):
             return []
 
         tokens = message.get(TOKENS_NAMES[TEXT])
diff --git a/rasa/nlu/selectors/response_selector.py b/rasa/nlu/selectors/response_selector.py
index c5c925ef99fa..8326bbce9f98 100644
--- a/rasa/nlu/selectors/response_selector.py
+++ b/rasa/nlu/selectors/response_selector.py
@@ -526,7 +526,7 @@ def process(self, messages: List[Message]) -> List[Message]:
             the associated intent_response_key and its similarity to the input.
         """
         for message in messages:
-            out = self._predict(message)
+            out = self._predict(message) if message.get(TEXT) else None
             top_label, label_ranking = self._predict_label(out)
 
             # Get the exact intent_response_key and the associated
diff --git a/tests/nlu/classifiers/test_diet_classifier.py b/tests/nlu/classifiers/test_diet_classifier.py
index c9a49b80f59b..9a6576749042 100644
--- a/tests/nlu/classifiers/test_diet_classifier.py
+++ b/tests/nlu/classifiers/test_diet_classifier.py
@@ -422,6 +422,18 @@ async def test_softmax_normalization(
     assert parse_data.get("intent") == intent_ranking[0]
 
 
+async def test_process_empty_input(
+    create_train_load_and_process_diet: Callable[..., Message],
+):
+    message = create_train_load_and_process_diet(
+        diet_config={EPOCHS: 1}, message_text="", expect_intent=False
+    )
+    assert message.get(TEXT) == ""
+    assert not message.get(INTENT)["name"]
+    assert message.get(INTENT)["confidence"] == 0.0
+    assert not message.get(ENTITIES)
+
+
 async def test_margin_loss_is_not_normalized(
     create_train_load_and_process_diet: Callable[..., Message]
 ):
diff --git a/tests/nlu/classifiers/test_sklearn_classifier.py b/tests/nlu/classifiers/test_sklearn_classifier.py
index 23bf14293c59..868768491a6a 100644
--- a/tests/nlu/classifiers/test_sklearn_classifier.py
+++ b/tests/nlu/classifiers/test_sklearn_classifier.py
@@ -14,6 +14,8 @@
 from rasa.engine.storage.storage import ModelStorage
 from rasa.nlu.classifiers.sklearn_intent_classifier import SklearnIntentClassifier
 from rasa.shared.nlu.training_data.training_data import TrainingData
+from rasa.shared.nlu.constants import TEXT, INTENT
+from rasa.shared.nlu.training_data.message import Message
 
 
 @pytest.fixture()
@@ -70,6 +72,35 @@ def test_persist_and_load(
         assert m1.get("intent") == m2.get("intent")
 
 
+def test_process_empty_input(
+    training_data: TrainingData,
+    default_sklearn_intent_classifier: SklearnIntentClassifier,
+    default_model_storage: ModelStorage,
+    default_execution_context: ExecutionContext,
+    train_and_preprocess: Callable[..., Tuple[TrainingData, List[GraphComponent]]],
+    spacy_nlp_component: SpacyNLP,
+    spacy_model: SpacyModel,
+):
+    training_data = spacy_nlp_component.process_training_data(
+        training_data, spacy_model
+    )
+    training_data, loaded_pipeline = train_and_preprocess(
+        pipeline=[{"component": SpacyTokenizer}, {"component": SpacyFeaturizer},],
+        training_data=training_data,
+    )
+    default_sklearn_intent_classifier.train(training_data)
+    classifier = SklearnIntentClassifier.load(
+        SklearnIntentClassifier.get_default_config(),
+        default_model_storage,
+        Resource("sklearn"),
+        default_execution_context,
+    )
+    message = Message(data={TEXT: ""})
+    processed_message = classifier.process([message])[0]
+    assert processed_message.get(TEXT) == ""
+    assert not processed_message.get(INTENT)
+
+
 def test_loading_from_storage_fail(
     training_data: TrainingData,
     default_model_storage: ModelStorage,
diff --git a/tests/nlu/extractors/test_crf_entity_extractor.py b/tests/nlu/extractors/test_crf_entity_extractor.py
index 2fc6368e3edd..756f0a4c7215 100644
--- a/tests/nlu/extractors/test_crf_entity_extractor.py
+++ b/tests/nlu/extractors/test_crf_entity_extractor.py
@@ -170,6 +170,16 @@ async def test_train_persist_with_different_configurations(
     assert detected_entities[0]["value"] == "italian"
 
 
+def test_process_empty_input(
+    crf_entity_extractor: Callable[[Dict[Text, Any]], CRFEntityExtractor],
+):
+    crf_extractor = crf_entity_extractor({})
+    message = Message(data={TEXT: ""})
+    processed_message = crf_extractor.process([message])[0]
+    assert processed_message.get(TEXT) == ""
+    assert processed_message.get(ENTITIES) == []
+
+
 def test_crf_use_dense_features(
     crf_entity_extractor: Callable[[Dict[Text, Any]], CRFEntityExtractor],
     spacy_nlp: Any,
diff --git a/tests/nlu/selectors/test_selectors.py b/tests/nlu/selectors/test_selectors.py
index d08f5b2054e6..7ad77e7c3b54 100644
--- a/tests/nlu/selectors/test_selectors.py
+++ b/tests/nlu/selectors/test_selectors.py
@@ -615,6 +615,34 @@ def test_transformer_size_gets_corrected(train_persist_load_with_different_setti
     assert selector.component_config[TRANSFORMER_SIZE] == DEFAULT_TRANSFORMER_SIZE
 
 
+async def test_process_empty_input(
+    create_response_selector: Callable[[Dict[Text, Any]], ResponseSelector],
+    train_and_preprocess: Callable[..., Tuple[TrainingData, List[GraphComponent]]],
+    process_message: Callable[..., Message],
+):
+    pipeline = [
+        {"component": WhitespaceTokenizer},
+        {"component": CountVectorsFeaturizer},
+    ]
+    training_data, loaded_pipeline = train_and_preprocess(
+        pipeline, "data/test_selectors"
+    )
+
+    response_selector = create_response_selector({EPOCHS: 1})
+    response_selector.train(training_data=training_data)
+
+    message = Message(data={TEXT: ""})
+    message = process_message(loaded_pipeline, message)
+
+    classified_message = response_selector.process([message])[0]
+    output = classified_message.get("response_selector").get("default").get("response")
+
+    assert classified_message.get(TEXT) == ""
+    assert not output.get("responses")
+    assert output.get("confidence") == 0.0
+    assert not output.get("intent_response_key")
+
+
 @pytest.mark.timeout(120)
 async def test_adjusting_layers_incremental_training(
     create_response_selector: Callable[[Dict[Text, Any]], ResponseSelector],


Code review comments:
diff_hunk: @@ -615,6 +615,34 @@ def test_transformer_size_gets_corrected(train_persist_load_with_different_setti
     assert selector.component_config[TRANSFORMER_SIZE] == DEFAULT_TRANSFORMER_SIZE
 
 
+async def test_process_empty_input(
+    create_response_selector: Callable[[Dict[Text, Any]], ResponseSelector],
+    train_and_preprocess: Callable[..., Tuple[TrainingData, List[GraphComponent]]],
+    process_message: Callable[..., Message],
+):
+    pipeline = [
+        {"component": WhitespaceTokenizer},
+        {"component": CountVectorsFeaturizer},
+    ]
+    training_data, loaded_pipeline = train_and_preprocess(
+        pipeline, "data/test_selectors"
+    )
+
+    response_selector = create_response_selector({EPOCHS: 1})
+    response_selector.train(training_data=training_data)
+
+    message = Message(data={TEXT: ""})
+    message = process_message(loaded_pipeline, message)
+
+    classified_message = response_selector.process([message])[0]
+    output = classified_message.get("response_selector").get("default").get("response")
+
+    assert classified_message.get(TEXT) == ""
+    assert not output.get("responses")
+    assert output.get("confidence") == 0.0
+    assert not output.get("intent_response_key")
path: tests/nlu/selectors/test_selectors.py
text: @user1:
Please use existing constants for the strings over here. You'll find them being used in `process` method of `ResponseSelector` class as well.

@author:
forgot to change this
wIll do now