Repository: RasaHQ/rasa
Pull request: add full retrieval intent name
**Proposed changes**:
- add full retrieval intent name to tracker 
- closes https://github.com/RasaHQ/rasa/issues/4826

**Status (please check what you already did)**:
- [x] added some tests for the functionality
- [ ] updated the documentation
- [x] updated the changelog (please check [changelog](https://github.com/RasaHQ/rasa/tree/master/changelog) for instructions)
- [x] reformat files using `black` (please check [Readme](https://github.com/RasaHQ/rasa#code-style) for instructions)


Problem statement:
Retrieval intent needs to store full intent name in tracker
**Description of Problem**:
The intent `ask_faq/python_version` is grouped into a single retriever event and placed in the tracker store as `ask_faq`. If you are using the TwoStageFallbackPolicy and the `ActionDefaultAskAffirmation` to further contextualize the conversation with the user, lack of the full intent name means you cannot determine the users intent.

**Overview of the Solution**:
Store the full intent name in the tracker store when using retrieval intents.

Hints from issue discussion:
it should be available in the `tracker.latest_message` right? that should contain the top predicted response for the retrieval action
@amn41 maybe so, but not sure that solves all the problems this issue causes:

> `ask_faq/python_version` is grouped into the single retriever intent and placed in the tracker store as `ask_faq`

This seems less than ideal, i’ve also been talking to a customer who ran into this. Their use case involves lots of FAQs, but they want to know which ones are being asked the most so they can turn those into further contextual conversations to continue to help the user. as of now it is impossible when looking in the tracker store to filter by the specific faq.

right, but that isn't the use case for retrieval actions. Retrieval actions are there to cover all the things where context _doesn't_ matter, like FAQs https://rasa.com/docs/rasa/dialogue-elements/completing-tasks/#simple-questions . If you need to handle it differently, it should be a separate intent 
The idea here is to be able to decide, out of these questions, which ones _should_ actually be handled differently, which they can't do without conversations with real users, to see which paths should have further information. I guess you could do that by having them all as separate intents, but it IMO feels like we're missing an easy win here by not letting you filter by the sub-intent
ok, I get it now! Yeah it would be great to have a way to find out which faqs should be "promoted" to real intents
I think that fits super well into "Moving from step 2 to step 3" dialogue as well -- first start with handling a bunch of faqs, then see what ones your users have follow-up questions for
cc @akelad ideas for moving from step 2 to 3
@rgstephens is this still happening?
not by Greg probably. Do you want to take it on instead? Or should this be on a different board?
@erohmensing do you want to do a code walkthrough on where I left this?

Commit message: Merge branch 'full_retrieval_intent' of github.com:RasaHQ/rasa into full_retrieval_intent

Patch to review:
diff --git a/changelog/4826.improvement.rst b/changelog/4826.improvement.rst
new file mode 100644
index 000000000000..c7a3b11b400f
--- /dev/null
+++ b/changelog/4826.improvement.rst
@@ -0,0 +1,4 @@
+Add full retrieval intent name to message data
+``ResponseSelector`` will now add the full retrieval intent name
+e.g. ``faq/which_version`` to the prediction, making it accessible
+from the tracker.
\ No newline at end of file
diff --git a/rasa/nlu/selectors/response_selector.py b/rasa/nlu/selectors/response_selector.py
index a7be78a303f3..b3c3564e568f 100644
--- a/rasa/nlu/selectors/response_selector.py
+++ b/rasa/nlu/selectors/response_selector.py
@@ -2,13 +2,16 @@
 
 import numpy as np
 import tensorflow as tf
+from pathlib import Path
 
 from typing import Any, Dict, Optional, Text, Tuple, Union, List, Type
 
+import rasa.utils.io as io_utils
 from rasa.nlu.config import InvalidConfigError
 from rasa.nlu.training_data import TrainingData, Message
 from rasa.nlu.components import Component
 from rasa.nlu.featurizers.featurizer import Featurizer
+from rasa.nlu.model import Metadata
 from rasa.nlu.classifiers.diet_classifier import (
     DIETClassifier,
     DIET,
@@ -66,9 +69,12 @@
 from rasa.nlu.constants import (
     RESPONSE,
     RESPONSE_SELECTOR_PROPERTY_NAME,
+    RESPONSE_KEY_ATTRIBUTE,
+    INTENT,
     DEFAULT_OPEN_UTTERANCE_TYPE,
     TEXT,
 )
+
 from rasa.utils.tensorflow.model_data import RasaModelData
 from rasa.utils.tensorflow.models import RasaModel
 
@@ -203,6 +209,7 @@ def __init__(
         index_label_id_mapping: Optional[Dict[int, Text]] = None,
         index_tag_id_mapping: Optional[Dict[int, Text]] = None,
         model: Optional[RasaModel] = None,
+        retrieval_intent_mapping: Optional[Dict[Text, Text]] = None,
     ) -> None:
 
         component_config = component_config or {}
@@ -211,6 +218,7 @@ def __init__(
         component_config[INTENT_CLASSIFICATION] = True
         component_config[ENTITY_RECOGNITION] = False
         component_config[BILOU_FLAG] = None
+        self.retrieval_intent_mapping = retrieval_intent_mapping or {}
 
         super().__init__(
             component_config, index_label_id_mapping, index_tag_id_mapping, model
@@ -231,6 +239,20 @@ def _check_config_parameters(self) -> None:
         super()._check_config_parameters()
         self._load_selector_params(self.component_config)
 
+    @staticmethod
+    def _create_retrieval_intent_mapping(
+        training_data: TrainingData,
+    ) -> Dict[Text, Text]:
+        """Create response_key dictionary"""
+
+        retrieval_intent_mapping = {}
+        for example in training_data.intent_examples:
+            retrieval_intent_mapping[
+                example.get(RESPONSE)
+            ] = f"{example.get(INTENT)}/{example.get(RESPONSE_KEY_ATTRIBUTE)}"
+
+        return retrieval_intent_mapping
+
     @staticmethod
     def _set_message_property(
         message: Message, prediction_dict: Dict[Text, Any], selector_key: Text
@@ -262,6 +284,9 @@ def preprocess_train_data(self, training_data: TrainingData) -> RasaModelData:
         label_id_index_mapping = self._label_id_index_mapping(
             training_data, attribute=RESPONSE
         )
+        self.retrieval_intent_mapping = self._create_retrieval_intent_mapping(
+            training_data
+        )
 
         if not label_id_index_mapping:
             # no labels are present to train
@@ -288,6 +313,7 @@ def process(self, message: Message, **kwargs: Any) -> None:
 
         out = self._predict(message)
         label, label_ranking = self._predict_label(out)
+        retrieval_intent_name = self.retrieval_intent_mapping.get(label.get("name"))
 
         selector_key = (
             self.retrieval_intent
@@ -299,10 +325,55 @@ def process(self, message: Message, **kwargs: Any) -> None:
             f"Adding following selector key to message property: {selector_key}"
         )
 
-        prediction_dict = {"response": label, "ranking": label_ranking}
+        prediction_dict = {
+            "response": label,
+            "ranking": label_ranking,
+            "full_retrieval_intent": retrieval_intent_name,
+        }
 
         self._set_message_property(message, prediction_dict, selector_key)
 
+    def persist(self, file_name: Text, model_dir: Text) -> Dict[Text, Any]:
+        """Persist this model into the passed directory.
+
+        Return the metadata necessary to load the model again.
+        """
+        super().persist(file_name, model_dir)
+
+        model_dir = Path(model_dir)
+
+        io_utils.json_pickle(
+            model_dir / f"{file_name}.retrieval_intent_mapping.pkl",
+            self.retrieval_intent_mapping,
+        )
+        return {"file": file_name}
+
+    @classmethod
+    def load(
+        cls,
+        meta: Dict[Text, Any],
+        model_dir: Text = None,
+        model_metadata: Metadata = None,
+        cached_component: Optional["ResponseSelector"] = None,
+        **kwargs: Any,
+    ) -> "ResponseSelector":
+        """Loads the trained model from the provided directory."""
+
+        model = super().load(
+            meta, model_dir, model_metadata, cached_component, **kwargs
+        )
+
+        file_name = meta.get("file")
+        model_dir = Path(model_dir)
+
+        retrieval_intent_mapping = io_utils.json_unpickle(
+            model_dir / f"{file_name}.retrieval_intent_mapping.pkl"
+        )
+
+        model.retrieval_intent_mapping = retrieval_intent_mapping
+
+        return model
+
 
 class DIET2DIET(DIET):
     def _check_data(self) -> None:
diff --git a/tests/nlu/selectors/test_selectors.py b/tests/nlu/selectors/test_selectors.py
index fbce352a6b8b..7862ef3a03e7 100644
--- a/tests/nlu/selectors/test_selectors.py
+++ b/tests/nlu/selectors/test_selectors.py
@@ -4,6 +4,7 @@
 from rasa.nlu.training_data import load_data
 from rasa.nlu.train import Trainer, Interpreter
 from rasa.utils.tensorflow.constants import EPOCHS
+from rasa.nlu.constants import RESPONSE_SELECTOR_PROPERTY_NAME
 
 
 @pytest.mark.parametrize(
@@ -33,6 +34,12 @@ def test_train_selector(pipeline, component_builder, tmpdir):
     assert trainer.pipeline
 
     loaded = Interpreter.load(persisted_path, component_builder)
+    parsed = loaded.parse("hello")
 
     assert loaded.pipeline
-    assert loaded.parse("hello") is not None
+    assert parsed is not None
+    assert (
+        parsed.get(RESPONSE_SELECTOR_PROPERTY_NAME)
+        .get("default")
+        .get("full_intent_name")
+    ) is not None


Code review comments:
diff_hunk: @@ -33,6 +34,12 @@ def test_train_selector(pipeline, component_builder, tmpdir):
     assert trainer.pipeline
 
     loaded = Interpreter.load(persisted_path, component_builder)
+    parsed = loaded.parse("hello")
 
     assert loaded.pipeline
-    assert loaded.parse("hello") is not None
+    assert parsed is not None
+    assert (
+        parsed.get(RESPONSE_SELECTOR_PROPERTY_NAME)
+        .get("default")
+        .get("full_intent_name")
path: tests/nlu/selectors/test_selectors.py
text: @user1:
```suggestion
        .get("full_retrieval_intent")
```