Repository: RasaHQ/rasa
Pull request: hide only rule dialogue turns
**Proposed changes**:
- closes https://github.com/RasaHQ/rasa/issues/7226

originally I had 2 ideas: operate on states or use forgetting functionality in tracker.applied_events through specialized events. In the end I used the mix of these approaches.

**Status (please check what you already did)**:
- [x] added some tests for the functionality
- [x] updated the documentation
- [x] updated the changelog (please check [changelog](https://github.com/RasaHQ/rasa/tree/master/changelog) for instructions)
- [x] reformat files using `black` (please check [Readme](https://github.com/RasaHQ/rasa#code-style) for instructions)


Problem statement:
Unfeaturize rule snippets for inference through TED
**Description of Problem**:
Currently, during inference to predict the next bot action, the first check is to see if a rule can be triggered. If not, then the prediction is done through ML policies. However, this can potentially cause a problem because these rule snippets may not be part of any story during training and hence during inference the ML policy may be confused by their presence in the tracker. 

**Overview of the Solution**:
We can unfeaturize/revert conversation turns in the tracker which **only** appear as a rule in the training data. That way any rule which was never a part of a training story but is triggered at inference time is not part of the tracker when an ML policy like TED makes a prediction.

For example, we can have a rule -
```
intent: greet
action: utter_greet
```

Let’s say this rule doesn’t appear in any stories. The conversation may proceed as -
```
User: greet
Bot: utter_greet
User: mood_happy
Bot: ...
```
When the bot has to predict the next action after `mood_happy`, we unfeaturize the states corresponding to the conversation turn involving `greet` and `utter_greet`.

One way to do this is to add new event called `RuleReverted` which contains appropriate number of dialogue turns to revert. When `RulePolicy` is trained, it "learns" deterministically from data whether or not to predict this event during inference.

Hints from issue discussion:
I just understood, you cannot do it in `tracker.applied_events`, because rule policy should see this events. Rather, you should create a temporary tracker inside predict ted, and rule policy should "learn" whether this predicted action should be unfeaturized or not. This information (rather logical not) can be extracted from checking rule predictions on stories. Then `RulePolicy` can release optional event

Commit message: create featurize for prediction helpers to automatically set for_only_ml_policy

Patch to review:
diff --git a/changelog/7701.improvement.md b/changelog/7701.improvement.md
new file mode 100644
index 000000000000..13e4d85285fd
--- /dev/null
+++ b/changelog/7701.improvement.md
@@ -0,0 +1,3 @@
+Hide dialogue turns predicted by `RulePolicy` in the tracker states
+for ML-only policies like `TEDPolicy`
+if those dialogue turns only appear as rules in the training data and do not appear in stories.
diff --git a/docs/docs/rules.mdx b/docs/docs/rules.mdx
index 2af748bd1b9e..b72c3f084e3c 100644
--- a/docs/docs/rules.mdx
+++ b/docs/docs/rules.mdx
@@ -47,6 +47,13 @@ rules:
 This example rule applies at the start of conversation as well as when the user decides
 to a send a message with an intent `greet` in the middle of an ongoing conversation.
 
+Dialogue turns that only appear as rules in the training data and do not appear in stories
+will be ignored by ML-only policies like `TEDPolicy` at prediction time.
+For example if you define the greeting rule as above and don't add it to any of your stories,
+after `RulePolicy` predicts `utter_greet`, `TEDPolicy` will
+make predictions as if the `greet, utter_greet` turn did not occur.
+
+
 ### Rules for the Conversation Start
 
 To write a rule which only applies at the beginning of a conversation, add a
diff --git a/rasa/core/featurizers/tracker_featurizers.py b/rasa/core/featurizers/tracker_featurizers.py
index 26560266f440..b4ba43180923 100644
--- a/rasa/core/featurizers/tracker_featurizers.py
+++ b/rasa/core/featurizers/tracker_featurizers.py
@@ -51,17 +51,21 @@ def __init__(
         self.state_featurizer = state_featurizer
 
     @staticmethod
-    def _create_states(tracker: DialogueStateTracker, domain: Domain) -> List[State]:
+    def _create_states(
+        tracker: DialogueStateTracker, domain: Domain, for_only_ml_policy: bool = False
+    ) -> List[State]:
         """Create states for the given tracker.
 
         Args:
             tracker: a :class:`rasa.core.trackers.DialogueStateTracker`
             domain: a :class:`rasa.shared.core.domain.Domain`
+            for_only_ml_policy: If True ignore dialogue turns that are present
+                only in rules.
 
         Returns:
             a list of states
         """
-        return tracker.past_states(domain)
+        return tracker.past_states(domain, for_only_ml_policy)
 
     def _featurize_states(
         self,
@@ -235,14 +239,17 @@ def prediction_states(
         trackers: List[DialogueStateTracker],
         domain: Domain,
         use_text_for_last_user_input: bool = False,
+        for_only_ml_policy: bool = False,
     ) -> List[List[State]]:
         """Transforms list of trackers to lists of states for prediction.
 
         Args:
-            trackers: The trackers to transform
-            domain: The domain
+            trackers: The trackers to transform.
+            domain: The domain.
             use_text_for_last_user_input: Indicates whether to use text or intent label
                 for featurizing last user input.
+            for_only_ml_policy: If True ignore dialogue turns that are present
+                only in rules.
 
         Returns:
             A list of states.
@@ -257,6 +264,7 @@ def create_state_features(
         domain: Domain,
         interpreter: NaturalLanguageInterpreter,
         use_text_for_last_user_input: bool = False,
+        for_only_ml_policy: bool = False,
     ) -> List[List[Dict[Text, List["Features"]]]]:
         """Create state features for prediction.
 
@@ -266,6 +274,8 @@ def create_state_features(
             interpreter: The interpreter
             use_text_for_last_user_input: Indicates whether to use text or intent label
                 for featurizing last user input.
+            for_only_ml_policy: If True ignore dialogue turns that are present
+                only in rules.
 
         Returns:
             A dictionary of state type (INTENT, TEXT, ACTION_NAME, ACTION_TEXT,
@@ -273,7 +283,7 @@ def create_state_features(
             turns in all trackers.
         """
         trackers_as_states = self.prediction_states(
-            trackers, domain, use_text_for_last_user_input
+            trackers, domain, use_text_for_last_user_input, for_only_ml_policy
         )
         return self._featurize_states(trackers_as_states, interpreter)
 
@@ -393,20 +403,24 @@ def prediction_states(
         trackers: List[DialogueStateTracker],
         domain: Domain,
         use_text_for_last_user_input: bool = False,
+        for_only_ml_policy: bool = False,
     ) -> List[List[State]]:
         """Transforms list of trackers to lists of states for prediction.
 
         Args:
-            trackers: The trackers to transform
-            domain: The domain,
+            trackers: The trackers to transform.
+            domain: The domain.
             use_text_for_last_user_input: Indicates whether to use text or intent label
                 for featurizing last user input.
+            for_only_ml_policy: If True ignore dialogue turns that are present
+                only in rules.
 
         Returns:
             A list of states.
         """
         trackers_as_states = [
-            self._create_states(tracker, domain) for tracker in trackers
+            self._create_states(tracker, domain, for_only_ml_policy)
+            for tracker in trackers
         ]
         self._choose_last_user_input(trackers_as_states, use_text_for_last_user_input)
 
@@ -548,20 +562,24 @@ def prediction_states(
         trackers: List[DialogueStateTracker],
         domain: Domain,
         use_text_for_last_user_input: bool = False,
+        for_only_ml_policy: bool = False,
     ) -> List[List[State]]:
         """Transforms list of trackers to lists of states for prediction.
 
         Args:
-            trackers: The trackers to transform
-            domain: The domain
+            trackers: The trackers to transform.
+            domain: The domain.
             use_text_for_last_user_input: Indicates whether to use text or intent label
                 for featurizing last user input.
+            for_only_ml_policy: If True ignore dialogue turns that are present
+                only in rules.
 
         Returns:
             A list of states.
         """
         trackers_as_states = [
-            self._create_states(tracker, domain) for tracker in trackers
+            self._create_states(tracker, domain, for_only_ml_policy)
+            for tracker in trackers
         ]
         trackers_as_states = [
             self.slice_state_history(states, self.max_history)
diff --git a/rasa/core/policies/form_policy.py b/rasa/core/policies/form_policy.py
index 5fcb2f7a2db1..95fd234608c0 100644
--- a/rasa/core/policies/form_policy.py
+++ b/rasa/core/policies/form_policy.py
@@ -112,8 +112,7 @@ def state_is_unhappy(self, tracker: DialogueStateTracker, domain: Domain) -> boo
         # since it is assumed that training stories contain
         # only unhappy paths, notify the form that
         # it should not be validated if predicted by other policy
-        tracker_as_states = self.featurizer.prediction_states([tracker], domain)
-        states = tracker_as_states[0]
+        states = self.prediction_states(tracker, domain)
 
         memorized_form = self.recall(states, tracker, domain)
 
diff --git a/rasa/core/policies/memoization.py b/rasa/core/policies/memoization.py
index 849678e3f4a1..55fa21efeb7f 100644
--- a/rasa/core/policies/memoization.py
+++ b/rasa/core/policies/memoization.py
@@ -224,8 +224,7 @@ def predict_action_probabilities(
     ) -> PolicyPrediction:
         result = self._default_predictions(domain)
 
-        tracker_as_states = self.featurizer.prediction_states([tracker], domain)
-        states = tracker_as_states[0]
+        states = self.prediction_states(tracker, domain)
         logger.debug(f"Current tracker state:{self.format_tracker_states(states)}")
         predicted_action_name = self.recall(states, tracker, domain)
         if predicted_action_name is not None:
@@ -309,10 +308,7 @@ def _recall_using_delorean(self, old_states, tracker, domain) -> Optional[Text]:
 
         mcfly_tracker = self._back_to_the_future(tracker)
         while mcfly_tracker is not None:
-            tracker_as_states = self.featurizer.prediction_states(
-                [mcfly_tracker], domain
-            )
-            states = tracker_as_states[0]
+            states = self.prediction_states(mcfly_tracker, domain)
 
             if old_states != states:
                 # check if we like new futures
diff --git a/rasa/core/policies/policy.py b/rasa/core/policies/policy.py
index 0294705baa33..11dc0eabdfa5 100644
--- a/rasa/core/policies/policy.py
+++ b/rasa/core/policies/policy.py
@@ -23,7 +23,7 @@
 import rasa.shared.utils.common
 import rasa.utils.common
 import rasa.shared.utils.io
-from rasa.shared.core.domain import Domain
+from rasa.shared.core.domain import Domain, State
 from rasa.core.featurizers.single_state_featurizer import SingleStateFeaturizer
 from rasa.core.featurizers.tracker_featurizers import (
     TrackerFeaturizer,
@@ -36,7 +36,6 @@
 from rasa.core.constants import DEFAULT_POLICY_PRIORITY
 from rasa.shared.core.constants import USER, SLOTS, PREVIOUS_ACTION, ACTIVE_LOOP
 from rasa.shared.nlu.constants import ENTITIES, INTENT, TEXT, ACTION_TEXT, ACTION_NAME
-from rasa.utils.tensorflow.constants import EPOCHS
 
 if TYPE_CHECKING:
     from rasa.shared.nlu.training_data.features import Features
@@ -193,6 +192,62 @@ def featurize_for_training(
 
         return state_features, label_ids, entity_tags
 
+    def prediction_states(
+        self,
+        tracker: DialogueStateTracker,
+        domain: Domain,
+        use_text_for_last_user_input: bool = False,
+    ) -> List[State]:
+        """Transforms list of trackers to lists of states for prediction.
+
+        Args:
+            tracker: The tracker to be featurized.
+            domain: The Domain.
+            use_text_for_last_user_input: Indicates whether to use text or intent label
+                for featurizing last user input.
+
+        Returns:
+            A list of states.
+        """
+        return self.featurizer.prediction_states(
+            [tracker],
+            domain,
+            use_text_for_last_user_input=use_text_for_last_user_input,
+            for_only_ml_policy=self.supported_data() == SupportedData.ML_DATA,
+        )[0]
+
+    def featurize_for_prediction(
+        self,
+        tracker: DialogueStateTracker,
+        domain: Domain,
+        interpreter: NaturalLanguageInterpreter,
+        use_text_for_last_user_input: bool = False,
+    ) -> List[List[Dict[Text, List["Features"]]]]:
+        """Transform training trackers into a vector representation.
+
+        The trackers, consisting of multiple turns, will be transformed
+        into a float vector which can be used by a ML model.
+
+        Args:
+            tracker: The tracker to be featurized.
+            domain: The Domain.
+            interpreter: The NLU interpreter.
+            use_text_for_last_user_input: Indicates whether to use text or intent label
+                for featurizing last user input.
+
+        Returns:
+            A dictionary of attribute (INTENT, TEXT, ACTION_NAME, ACTION_TEXT,
+            ENTITIES, SLOTS, FORM) to a list of features for all dialogue turns in
+            the tracker.
+        """
+        return self.featurizer.create_state_features(
+            [tracker],
+            domain,
+            interpreter,
+            use_text_for_last_user_input=use_text_for_last_user_input,
+            for_only_ml_policy=self.supported_data() == SupportedData.ML_DATA,
+        )
+
     def train(
         self,
         training_trackers: List[TrackerWithCachedStates],
diff --git a/rasa/core/policies/rule_policy.py b/rasa/core/policies/rule_policy.py
index 4d056569a09c..10ddd509f04c 100644
--- a/rasa/core/policies/rule_policy.py
+++ b/rasa/core/policies/rule_policy.py
@@ -9,7 +9,12 @@
 from rasa.shared.constants import DOCS_URL_RULES
 from rasa.shared.exceptions import RasaException
 import rasa.shared.utils.io
-from rasa.shared.core.events import LoopInterrupted, UserUttered, ActionExecuted
+from rasa.shared.core.events import (
+    LoopInterrupted,
+    UserUttered,
+    ActionExecuted,
+    HideRuleTurn,
+)
 from rasa.core.featurizers.tracker_featurizers import TrackerFeaturizer
 from rasa.shared.nlu.interpreter import NaturalLanguageInterpreter
 from rasa.core.policies.memoization import MemoizationPolicy
@@ -58,12 +63,15 @@
 
 RULES = "rules"
 RULES_FOR_LOOP_UNHAPPY_PATH = "rules_for_loop_unhappy_path"
+RULES_NOT_IN_STORIES = "rules_not_in_stories"
+ONLY_RULE_SLOTS = "only_rule_slots"
+ONLY_RULE_LOOPS = "only_rule_loops"
 
 LOOP_WAS_INTERRUPTED = "loop_was_interrupted"
 DO_NOT_PREDICT_LOOP_ACTION = "do_not_predict_loop_action"
 
-DEFAULT_RULES = "predicting default action"
-LOOP_RULES = "handling active loops and forms"
+DEFAULT_RULES = "predicting default action with intent "
+LOOP_RULES = "handling active loops and forms - "
 
 
 class InvalidRule(RasaException):
@@ -404,6 +412,38 @@ def _check_for_incomplete_rules(
 
         logger.debug("Found no incompletions in rules.")
 
+    @staticmethod
+    def _get_slots_loops_from_states(
+        trackers_as_states: List[List[State]],
+    ) -> Tuple[Set[Text], Set[Text]]:
+        slots = set()
+        loops = set()
+        for states in trackers_as_states:
+            for state in states:
+                slots.update(set(state.get(SLOTS, {}).keys()))
+                active_loop = state.get(ACTIVE_LOOP, {}).get(LOOP_NAME)
+                if active_loop:
+                    loops.add(active_loop)
+        return slots, loops
+
+    def _find_only_rule_slots_loops(
+        self,
+        rule_trackers_as_states: List[List[State]],
+        story_trackers_as_states: List[List[State]],
+    ) -> Tuple[List[Text], List[Text]]:
+        rule_slots, rule_loops = self._get_slots_loops_from_states(
+            rule_trackers_as_states
+        )
+        story_slots, story_loops = self._get_slots_loops_from_states(
+            story_trackers_as_states
+        )
+
+        # set is not json serializable, so convert to list
+        return (
+            list(rule_slots - story_slots - {SHOULD_NOT_BE_SET}),
+            list(rule_loops - story_loops - {SHOULD_NOT_BE_SET}),
+        )
+
     def _predict_next_action(
         self,
         tracker: TrackerWithCachedStates,
@@ -426,15 +466,13 @@ def _predict_next_action(
 
         return predicted_action_name
 
-    def _check_prediction(
+    def _predicted_action_name(
         self,
         tracker: TrackerWithCachedStates,
         domain: Domain,
         interpreter: NaturalLanguageInterpreter,
         gold_action_name: Text,
-        collect_sources: bool,
     ) -> Optional[Text]:
-
         predicted_action_name = self._predict_next_action(tracker, domain, interpreter)
         # if there is an active_loop,
         # RulePolicy will always predict active_loop first,
@@ -449,22 +487,51 @@ def _check_prediction(
                 tracker, domain, interpreter
             )
 
-        if collect_sources:
-            # we need to remember which action should be predicted by the rule
-            # in order to correctly output the names of the contradicting rules
-            rule_name = tracker.sender_id
-            if self._prediction_source in {DEFAULT_RULES, LOOP_RULES}:
-                # the real gold action contradict the one in the rules in this case
-                gold_action_name = predicted_action_name
-                rule_name = self._prediction_source
-
-            self._rules_sources[self._prediction_source].append(
-                (rule_name, gold_action_name)
-            )
-            return
+        return predicted_action_name
+
+    def _collect_sources(
+        self,
+        tracker: TrackerWithCachedStates,
+        predicted_action_name: Optional[Text],
+        gold_action_name: Text,
+    ) -> None:
+        # we need to remember which action should be predicted by the rule
+        # in order to correctly output the names of the contradicting rules
+        rule_name = tracker.sender_id
+        if self._prediction_source.startswith(
+            DEFAULT_RULES
+        ) or self._prediction_source.startswith(LOOP_RULES):
+            # the real gold action contradict the one in the rules in this case
+            gold_action_name = predicted_action_name
+            rule_name = self._prediction_source
+
+        self._rules_sources[self._prediction_source].append(
+            (rule_name, gold_action_name)
+        )
+
+    @staticmethod
+    def _default_sources() -> Set[Text]:
+        return {
+            DEFAULT_RULES + default_intent
+            for default_intent in DEFAULT_ACTION_MAPPINGS.keys()
+        }
+
+    @staticmethod
+    def _handling_loop_sources(domain: Domain) -> Set[Text]:
+        loop_sources = set()
+        for loop_name in domain.form_names:
+            loop_sources.add(LOOP_RULES + loop_name)
+            loop_sources.add(LOOP_RULES + loop_name + " - " + ACTION_LISTEN_NAME)
+        return loop_sources
 
+    def _check_prediction(
+        self,
+        tracker: TrackerWithCachedStates,
+        predicted_action_name: Optional[Text],
+        gold_action_name: Text,
+    ) -> List[Text]:
         if not predicted_action_name or predicted_action_name == gold_action_name:
-            return
+            return []
 
         tracker_type = "rule" if tracker.is_rule_tracker else "story"
         contradicting_rules = {
@@ -482,7 +549,7 @@ def _check_prediction(
         if predicted_action_name != self._fallback_action_name:
             error_message += f" which predicted action '{predicted_action_name}'"
 
-        return error_message + "."
+        return [error_message + "."]
 
     def _run_prediction_on_trackers(
         self,
@@ -490,11 +557,12 @@ def _run_prediction_on_trackers(
         domain: Domain,
         interpreter: NaturalLanguageInterpreter,
         collect_sources: bool,
-    ) -> List[Text]:
+    ) -> Tuple[List[Text], Set[Text]]:
         if collect_sources:
             self._rules_sources = defaultdict(list)
 
         error_messages = []
+        rules_used_in_stories = set()
         pbar = tqdm(
             trackers,
             desc="Processed trackers",
@@ -524,19 +592,30 @@ def _run_prediction_on_trackers(
                     continue
 
                 gold_action_name = event.action_name or event.action_text
-                error_message = self._check_prediction(
-                    running_tracker,
-                    domain,
-                    interpreter,
-                    gold_action_name,
-                    collect_sources,
+                predicted_action_name = self._predicted_action_name(
+                    running_tracker, domain, interpreter, gold_action_name
                 )
-                if error_message:
-                    error_messages.append(error_message)
+                if collect_sources:
+                    self._collect_sources(
+                        running_tracker, predicted_action_name, gold_action_name
+                    )
+                else:
+                    # to be able to remove only rules turns from the dialogue history
+                    # for ML policies,
+                    # we need to know which rules were used in ML trackers
+                    if (
+                        not tracker.is_rule_tracker
+                        and predicted_action_name == gold_action_name
+                    ):
+                        rules_used_in_stories.add(self._prediction_source)
+
+                    error_messages += self._check_prediction(
+                        running_tracker, predicted_action_name, gold_action_name
+                    )
 
                 running_tracker.update(event)
 
-        return error_messages
+        return error_messages, rules_used_in_stories
 
     def _find_contradicting_rules(
         self,
@@ -544,7 +623,7 @@ def _find_contradicting_rules(
         all_trackers: List[TrackerWithCachedStates],
         domain: Domain,
         interpreter: NaturalLanguageInterpreter,
-    ) -> None:
+    ) -> List[Text]:
         logger.debug("Started checking rules and stories for contradictions.")
         # during training we run `predict_action_probabilities` to check for
         # contradicting rules.
@@ -553,11 +632,11 @@ def _find_contradicting_rules(
         logger.setLevel(logging.WARNING)
 
         # we need to run prediction on rule trackers twice, because we need to collect
-        # information which rule snippets contributed to the learned rules
+        # the information about which rule snippets contributed to the learned rules
         self._run_prediction_on_trackers(
             rule_trackers, domain, interpreter, collect_sources=True
         )
-        error_messages = self._run_prediction_on_trackers(
+        error_messages, rules_used_in_stories = self._run_prediction_on_trackers(
             all_trackers, domain, interpreter, collect_sources=False
         )
 
@@ -571,29 +650,20 @@ def _find_contradicting_rules(
             )
 
         logger.debug("Found no contradicting rules.")
+        all_rules = (
+            set(self._rules_sources.keys())
+            | self._default_sources()
+            | self._handling_loop_sources(domain)
+        )
+        # set is not json serializable, so convert to list
+        return list(all_rules - rules_used_in_stories)
 
-    def train(
+    def _create_lookup_from_trackers(
         self,
-        training_trackers: List[TrackerWithCachedStates],
+        rule_trackers: List[TrackerWithCachedStates],
+        story_trackers: List[TrackerWithCachedStates],
         domain: Domain,
-        interpreter: NaturalLanguageInterpreter,
-        **kwargs: Any,
     ) -> None:
-
-        # only consider original trackers (no augmented ones)
-        training_trackers = [
-            t
-            for t in training_trackers
-            if not hasattr(t, "is_augmented") or not t.is_augmented
-        ]
-        # only use trackers from rule-based training data
-        rule_trackers = [t for t in training_trackers if t.is_rule_tracker]
-        if self._restrict_rules:
-            self._check_rule_restriction(rule_trackers)
-
-        if self._check_for_contradictions:
-            self._check_for_incomplete_rules(rule_trackers, domain)
-
         (
             rule_trackers_as_states,
             rule_trackers_as_actions,
@@ -604,12 +674,19 @@ def train(
         )
         self.lookup[RULES] = self._remove_rule_snippet_predictions(rules_lookup)
 
-        story_trackers = [t for t in training_trackers if not t.is_rule_tracker]
         (
             story_trackers_as_states,
             story_trackers_as_actions,
         ) = self.featurizer.training_states_and_actions(story_trackers, domain)
 
+        if self._check_for_contradictions:
+            (
+                self.lookup[ONLY_RULE_SLOTS],
+                self.lookup[ONLY_RULE_LOOPS],
+            ) = self._find_only_rule_slots_loops(
+                rule_trackers_as_states, story_trackers_as_states
+            )
+
         # use all trackers to find negative rules in unhappy paths
         trackers_as_states = rule_trackers_as_states + story_trackers_as_states
         trackers_as_actions = rule_trackers_as_actions + story_trackers_as_actions
@@ -621,11 +698,43 @@ def train(
             trackers_as_states, trackers_as_actions
         )
 
+    def train(
+        self,
+        training_trackers: List[TrackerWithCachedStates],
+        domain: Domain,
+        interpreter: NaturalLanguageInterpreter,
+        **kwargs: Any,
+    ) -> None:
+        """Trains the policy on given training trackers.
+
+        Args:
+            training_trackers: The list of the trackers.
+            domain: The domain.
+            interpreter: Interpreter which can be used by the polices for featurization.
+        """
+        # only consider original trackers (no augmented ones)
+        training_trackers = [
+            t
+            for t in training_trackers
+            if not hasattr(t, "is_augmented") or not t.is_augmented
+        ]
+        # trackers from rule-based training data
+        rule_trackers = [t for t in training_trackers if t.is_rule_tracker]
+        if self._restrict_rules:
+            self._check_rule_restriction(rule_trackers)
+        if self._check_for_contradictions:
+            self._check_for_incomplete_rules(rule_trackers, domain)
+
+        # trackers from ML-based training data
+        story_trackers = [t for t in training_trackers if not t.is_rule_tracker]
+
+        self._create_lookup_from_trackers(rule_trackers, story_trackers, domain)
+
         # make this configurable because checking might take a lot of time
         if self._check_for_contradictions:
             # using trackers here might not be the most efficient way, however
             # it allows us to directly test `predict_action_probabilities` method
-            self._find_contradicting_rules(
+            self.lookup[RULES_NOT_IN_STORIES] = self._find_contradicting_rules(
                 rule_trackers, training_trackers, domain, interpreter
             )
 
@@ -725,12 +834,12 @@ def _get_possible_keys(
     @staticmethod
     def _find_action_from_default_actions(
         tracker: DialogueStateTracker,
-    ) -> Optional[Text]:
+    ) -> Tuple[Optional[Text], Optional[Text]]:
         if (
             not tracker.latest_action_name == ACTION_LISTEN_NAME
             or not tracker.latest_message
         ):
-            return None
+            return None, None
 
         default_action_name = DEFAULT_ACTION_MAPPINGS.get(
             tracker.latest_message.intent.get(INTENT_NAME_KEY)
@@ -738,13 +847,17 @@ def _find_action_from_default_actions(
 
         if default_action_name:
             logger.debug(f"Predicted default action '{default_action_name}'.")
+            return (
+                default_action_name,
+                DEFAULT_RULES + tracker.latest_message.intent.get(INTENT_NAME_KEY),
+            )
 
-        return default_action_name
+        return None, None
 
     @staticmethod
     def _find_action_from_loop_happy_path(
         tracker: DialogueStateTracker,
-    ) -> Optional[Text]:
+    ) -> Tuple[Optional[Text], Optional[Text]]:
 
         active_loop_name = tracker.active_loop_name
         active_loop_rejected = tracker.active_loop.get(LOOP_REJECTED)
@@ -761,14 +874,19 @@ def _find_action_from_loop_happy_path(
 
         if should_predict_loop:
             logger.debug(f"Predicted loop '{active_loop_name}'.")
-            return active_loop_name
+            return active_loop_name, LOOP_RULES + active_loop_name
 
         # predict `action_listen` if loop action was run successfully
         if should_predict_listen:
             logger.debug(
                 f"Predicted '{ACTION_LISTEN_NAME}' after loop '{active_loop_name}'."
             )
-            return ACTION_LISTEN_NAME
+            return (
+                ACTION_LISTEN_NAME,
+                LOOP_RULES + active_loop_name + " - " + ACTION_LISTEN_NAME,
+            )
+
+        return None, None
 
     def _find_action_from_rules(
         self,
@@ -798,10 +916,7 @@ def _find_action_from_rules(
             # the text or the intent
             return None, None, False
 
-        tracker_as_states = self.featurizer.prediction_states(
-            [tracker], domain, use_text_for_last_user_input
-        )
-        states = tracker_as_states[0]
+        states = self.prediction_states(tracker, domain, use_text_for_last_user_input)
 
         current_states = self.format_tracker_states(states)
         logger.debug(f"Current tracker state:{current_states}")
@@ -850,7 +965,11 @@ def _find_action_from_rules(
                         f"Predicted loop '{active_loop_name}' by overwriting "
                         f"'{ACTION_LISTEN_NAME}' predicted by general rule."
                     )
-                    return active_loop_name, LOOP_RULES, returning_from_unhappy_path
+                    return (
+                        active_loop_name,
+                        LOOP_RULES + active_loop_name,
+                        returning_from_unhappy_path,
+                    )
 
                 # do not predict anything
                 predicted_action_name = None
@@ -896,13 +1015,15 @@ def predict_action_probabilities(
         # Rasa Open Source default actions overrule anything. If users want to achieve
         # the same, they need to write a rule or make sure that their loop rejects
         # accordingly.
-        default_action_name = self._find_action_from_default_actions(tracker)
+        (
+            default_action_name,
+            self._prediction_source,
+        ) = self._find_action_from_default_actions(tracker)
 
         # text has priority over intents including default,
         # however loop happy path has priority over rules prediction
         if default_action_name and not rules_action_name_from_text:
-            self._prediction_source = DEFAULT_RULES
-            return self._prediction(
+            return self._rule_prediction(
                 self._prediction_result(default_action_name, tracker, domain)
             )
 
@@ -910,18 +1031,20 @@ def predict_action_probabilities(
         # The rules or any other prediction will be applied only if a loop was rejected.
         # If we are in a loop, and the loop didn't run previously or rejected, we can
         # simply force predict the loop.
-        loop_happy_path_action_name = self._find_action_from_loop_happy_path(tracker)
+        (
+            loop_happy_path_action_name,
+            self._prediction_source,
+        ) = self._find_action_from_loop_happy_path(tracker)
         if loop_happy_path_action_name:
-            self._prediction_source = LOOP_RULES
             # this prediction doesn't use user input
             # and happy user input anyhow should be ignored during featurization
-            return self._prediction(
+            return self._rule_prediction(
                 self._prediction_result(loop_happy_path_action_name, tracker, domain)
             )
 
         # predict rules from text first
         if rules_action_name_from_text:
-            return self._prediction_with_unhappy_path(
+            return self._rule_prediction(
                 self._prediction_result(rules_action_name_from_text, tracker, domain),
                 returning_from_unhappy_path=returning_from_unhappy_path_from_text,
                 is_end_to_end_prediction=True,
@@ -942,7 +1065,7 @@ def predict_action_probabilities(
         else:
             probabilities = self._default_predictions(domain)
 
-        return self._prediction_with_unhappy_path(
+        return self._rule_prediction(
             probabilities,
             returning_from_unhappy_path=(
                 # returning_from_unhappy_path is a negative condition,
@@ -953,15 +1076,23 @@ def predict_action_probabilities(
             is_end_to_end_prediction=False,
         )
 
-    def _prediction_with_unhappy_path(
+    def _rule_prediction(
         self,
         probabilities: List[float],
-        returning_from_unhappy_path: bool,
-        is_end_to_end_prediction: bool,
+        returning_from_unhappy_path: bool = False,
+        is_end_to_end_prediction: bool = False,
     ) -> "PolicyPrediction":
+        optional_events = []
+        if self._prediction_source in self.lookup.get(RULES_NOT_IN_STORIES, []):
+            # the prediction is based on rules that are not present in the stories
+            optional_events = [
+                HideRuleTurn(self.lookup[ONLY_RULE_SLOTS], self.lookup[ONLY_RULE_LOOPS])
+            ]
+
         return self._prediction(
             probabilities,
             events=[LoopInterrupted(True)] if returning_from_unhappy_path else [],
+            optional_events=optional_events,
             is_end_to_end_prediction=is_end_to_end_prediction,
         )
 
diff --git a/rasa/core/policies/sklearn_policy.py b/rasa/core/policies/sklearn_policy.py
index 2efcab236ce3..9f9acf98e9ff 100644
--- a/rasa/core/policies/sklearn_policy.py
+++ b/rasa/core/policies/sklearn_policy.py
@@ -284,7 +284,7 @@ def predict_action_probabilities(
         interpreter: NaturalLanguageInterpreter,
         **kwargs: Any,
     ) -> PolicyPrediction:
-        X = self.featurizer.create_state_features([tracker], domain, interpreter)
+        X = self.featurize_for_prediction(tracker, domain, interpreter)
         training_data, _ = model_data_utils.convert_to_data_format(
             X, self.zero_state_features
         )
diff --git a/rasa/core/policies/ted_policy.py b/rasa/core/policies/ted_policy.py
index f06530b26a54..3bf8f62f0e7f 100644
--- a/rasa/core/policies/ted_policy.py
+++ b/rasa/core/policies/ted_policy.py
@@ -538,8 +538,8 @@ def _featurize_tracker_for_e2e(
         # and second - an optional one (see conditions below),
         # the first example in the constructed batch either does not contain user input
         # or uses intent or text based on whether TED is e2e only.
-        tracker_state_features = self.featurizer.create_state_features(
-            [tracker], domain, interpreter, use_text_for_last_user_input=self.only_e2e
+        tracker_state_features = self.featurize_for_prediction(
+            tracker, domain, interpreter, use_text_for_last_user_input=self.only_e2e,
         )
         # the second - text, but only after user utterance and if not only e2e
         if (
@@ -547,13 +547,13 @@ def _featurize_tracker_for_e2e(
             and TEXT in self.fake_features
             and not self.only_e2e
         ):
-            tracker_state_features += self.featurizer.create_state_features(
-                [tracker], domain, interpreter, use_text_for_last_user_input=True
+            tracker_state_features += self.featurize_for_prediction(
+                tracker, domain, interpreter, use_text_for_last_user_input=True,
             )
         return tracker_state_features
 
     def _pick_confidence(
-        self, confidences: np.ndarray, similarities: np.ndarray
+        self, confidences: np.ndarray, similarities: np.ndarray, domain: Domain
     ) -> Tuple[np.ndarray, bool]:
         # the confidences and similarities have shape (batch-size x number of actions)
         # batch-size can only be 1 or 2;
@@ -568,16 +568,30 @@ def _pick_confidence(
             # we use similarities to pick appropriate input,
             # since it seems to be more accurate measure,
             # policy is trained to maximize the similarity not the confidence
+            non_e2e_action_name = domain.action_names_or_texts[
+                np.argmax(confidences[0])
+            ]
+            e2e_action_name = domain.action_names_or_texts[np.argmax(confidences[1])]
+            logger.debug(f"User intent lead to '{non_e2e_action_name}'.")
+            logger.debug(f"User text lead to '{e2e_action_name}'.")
             if (
                 np.max(confidences[1]) > self.config[E2E_CONFIDENCE_THRESHOLD]
                 # TODO maybe compare confidences is better
                 and np.max(similarities[1]) > np.max(similarities[0])
             ):
+                logger.debug(f"TED predicted '{e2e_action_name}' based on user text.")
                 return confidences[1], True
 
+            logger.debug(f"TED predicted '{non_e2e_action_name}' based on user intent.")
             return confidences[0], False
 
         # by default the first example in a batch is the one to use for prediction
+        predicted_action_name = domain.action_names_or_texts[np.argmax(confidences[0])]
+        basis_for_prediction = "text" if self.only_e2e else "intent"
+        logger.debug(
+            f"TED predicted '{predicted_action_name}' "
+            f"based on user {basis_for_prediction}."
+        )
         return confidences[0], self.only_e2e
 
     def predict_action_probabilities(
@@ -606,7 +620,9 @@ def predict_action_probabilities(
         similarities = output["similarities"].numpy()[:, -1, :]
         confidences = output["action_scores"].numpy()[:, -1, :]
         # take correct prediction from batch
-        confidence, is_e2e_prediction = self._pick_confidence(confidences, similarities)
+        confidence, is_e2e_prediction = self._pick_confidence(
+            confidences, similarities, domain
+        )
 
         if self.config[LOSS_TYPE] == SOFTMAX and self.config[RANKING_LENGTH] > 0:
             confidence = rasa.utils.train_utils.normalize(
@@ -1212,7 +1228,7 @@ def _encode_real_features_per_attribute(
         Args:
             tf_batch_data: dictionary mapping every attribute to its features and masks
             attribute: the attribute we will encode features for
-            (e.g., ACTION_NAME, INTENT)
+                (e.g., ACTION_NAME, INTENT)
 
         Returns:
             A tensor combining  all features for `attribute`
@@ -1312,10 +1328,9 @@ def _convert_to_original_shape(
 
         Args:
             attribute_features: the "real" features to convert
-            attribute_mask:  the tensor containing the position of "real" features
-                in the dialogue, shape is (batch-size x dialogue_len x 1)
-            dialogue_lengths: the tensor containing the actual dialogue length,
-                shape is (batch-size,)
+            tf_batch_data: dictionary mapping every attribute to its features and masks
+            attribute: the attribute we will encode features for
+                (e.g., ACTION_NAME, INTENT)
 
         Returns:
             The converted attribute features
diff --git a/rasa/shared/core/domain.py b/rasa/shared/core/domain.py
index afffdf79fed1..efeba3b48b74 100644
--- a/rasa/shared/core/domain.py
+++ b/rasa/shared/core/domain.py
@@ -1068,15 +1068,90 @@ def get_active_states(self, tracker: "DialogueStateTracker") -> State:
         }
         return self._clean_state(state)
 
+    @staticmethod
+    def _remove_only_rule_features(
+        state: State, tracker: "DialogueStateTracker"
+    ) -> None:
+        # remove only rule slots
+        for slot in tracker.only_rule_slots:
+            if state.get(rasa.shared.core.constants.SLOTS, {}).get(slot):
+                del state[rasa.shared.core.constants.SLOTS][slot]
+        # remove active loop feature if it is only rule loop
+        if (
+            state.get(rasa.shared.core.constants.ACTIVE_LOOP, {}).get(
+                rasa.shared.core.constants.LOOP_NAME
+            )
+            in tracker.only_rule_loops
+        ):
+            del state[rasa.shared.core.constants.ACTIVE_LOOP]
+
+    @staticmethod
+    def _substitute_only_rule_user_input(state: State, last_ml_state: State) -> None:
+        if not rasa.shared.core.trackers.is_prev_action_listen_in_state(state):
+            if not last_ml_state.get(rasa.shared.core.constants.USER) and state.get(
+                rasa.shared.core.constants.USER
+            ):
+                del state[rasa.shared.core.constants.USER]
+            elif last_ml_state.get(rasa.shared.core.constants.USER):
+                state[rasa.shared.core.constants.USER] = last_ml_state[
+                    rasa.shared.core.constants.USER
+                ]
+
     def states_for_tracker_history(
-        self, tracker: "DialogueStateTracker"
+        self, tracker: "DialogueStateTracker", for_only_ml_policy: bool = False
     ) -> List[State]:
-        """Array of states for each state of the trackers history."""
-        return [
-            self.get_active_states(tr) for tr in tracker.generate_all_prior_trackers()
-        ]
+        """Array of states for each state of the trackers history.
+
+        Args:
+            tracker: Instance of `DialogueStateTracker` to featurize.
+            for_only_ml_policy: If True ignore dialogue turns that are present
+                only in rules.
+
+        Return:
+            A list of states.
+        """
+        states = []
+        last_ml_action_sub_state = None
+        last_action_is_ml_action = True
+        for tr in tracker.generate_all_prior_trackers():
+            if for_only_ml_policy:
+                # remember previous ml action based on the last non hidden turn
+                # we need this to override previous action in the ml state
+                if last_action_is_ml_action:
+                    last_ml_action_sub_state = self._get_prev_action_sub_state(tr)
+                last_action_is_ml_action = not tr.hide_rule_turn
+
+                if tr.hide_rule_turn:
+                    continue
+
+            state = self.get_active_states(tr)
+
+            if for_only_ml_policy:
+                # clean state from only rule features
+                self._remove_only_rule_features(state, tracker)
+                # make sure user input is the same as for previous state
+                # for non action_listen turns
+                if states:
+                    self._substitute_only_rule_user_input(state, states[-1])
+                # substitute previous rule action with last_ml_action_sub_state
+                if last_ml_action_sub_state:
+                    state[
+                        rasa.shared.core.constants.PREVIOUS_ACTION
+                    ] = last_ml_action_sub_state
+
+            states.append(state)
+
+        return states
 
     def slots_for_entities(self, entities: List[Dict[Text, Any]]) -> List[SlotSet]:
+        """Find which slots should be auto fill with entities.
+
+        Args:
+            entities: The list of entities.
+
+        Returns:
+            A list of `SlotSet` events.
+        """
         if self.store_entities_as_slots:
             slot_events = []
             for s in self.slots:
diff --git a/rasa/shared/core/events.py b/rasa/shared/core/events.py
index be081e2038ad..df39dd2ec584 100644
--- a/rasa/shared/core/events.py
+++ b/rasa/shared/core/events.py
@@ -723,7 +723,10 @@ def __hash__(self) -> int:
 
     def __eq__(self, other: Any) -> bool:
         """Compares this event with another event."""
-        return isinstance(other, EntitiesAdded)
+        if not isinstance(other, EntitiesAdded):
+            return NotImplemented
+
+        return self.entities == other.entities
 
     @classmethod
     def _from_parameters(cls, parameters: Dict[Text, Any]) -> "EntitiesAdded":
@@ -1540,6 +1543,9 @@ def apply_to(self, tracker: "DialogueStateTracker") -> None:
         """Applies event to current conversation state."""
         tracker.set_latest_action(self.as_sub_state())
         tracker.clear_followup_action()
+        # HideRuleTurn event is added by RulePolicy before actual action is executed,
+        # so we can safely reset it on each action executed
+        tracker.hide_rule_turn = False
 
 
 class AgentUttered(SkipEventInMDStoryMixin):
@@ -1875,3 +1881,77 @@ def apply_to(self, tracker: "DialogueStateTracker") -> None:
         """Applies event to current conversation state."""
         # noinspection PyProtectedMember
         tracker._reset()
+
+
+class HideRuleTurn(SkipEventInMDStoryMixin, AlwaysEqualEventMixin):
+    """Emulate undoing of the last dialogue turn.
+
+    The bot reverts everything until before the most recent action.
+    This includes the action itself, as well as any events that
+    action created, like set slot events, unless they are in a list
+    to reapply them.
+    """
+
+    type_name = "hide_rule_turn"
+
+    def __init__(
+        self,
+        only_rule_slots: List[Text],
+        only_rule_loops: List[Text],
+        timestamp: Optional[float] = None,
+        metadata: Optional[Dict[Text, Any]] = None,
+    ) -> None:
+        """Initializes event.
+
+        Args:
+            only_rule_slots: The list of slot names,
+                SlotSet events for which shouldn't be hidden
+            only_rule_loops: The list of loop names,
+                ActiveLoop events for which shouldn't be hidden
+            timestamp: the timestamp
+            metadata: some optional metadata
+        """
+        super().__init__(timestamp, metadata)
+        self.only_rule_slots = only_rule_slots
+        self.only_rule_loops = only_rule_loops
+
+    def __hash__(self) -> int:
+        """Returns unique hash for event."""
+        return hash(32143124321)
+
+    @classmethod
+    def _from_parameters(cls, parameters: Dict[Text, Any]) -> "HideRuleTurn":
+        return HideRuleTurn(
+            parameters.get("only_rule_slots"),
+            parameters.get("only_rule_loops"),
+            parameters.get("timestamp"),
+            parameters.get("metadata"),
+        )
+
+    def as_dict(self) -> Dict[Text, Any]:
+        """Converts the event into a dict.
+
+        Returns:
+            A dict that represents this event.
+        """
+        d = super().as_dict()
+        d.update(
+            {
+                "only_rule_slots": self.only_rule_slots,
+                "only_rule_loops": self.only_rule_loops,
+            }
+        )
+        return d
+
+    def apply_to(self, tracker: "DialogueStateTracker") -> None:
+        """Applies event to current conversation state.
+
+        Args:
+            tracker: The current conversation state.
+        """
+        # HideRuleTurn event is added by RulePolicy before actual action is executed,
+        # we will reset it on each action executed
+        tracker.hide_rule_turn = True
+        # only rule slots and loops are always the same for all the trackers
+        tracker.only_rule_slots = self.only_rule_slots
+        tracker.only_rule_loops = self.only_rule_loops
diff --git a/rasa/shared/core/generator.py b/rasa/shared/core/generator.py
index 994ee52fedaf..10248cb700cc 100644
--- a/rasa/shared/core/generator.py
+++ b/rasa/shared/core/generator.py
@@ -107,7 +107,19 @@ def _unfreeze_states(frozen_states: Deque[FrozenState]) -> List[State]:
             for frozen_state in frozen_states
         ]
 
-    def past_states(self, domain: Domain) -> List[State]:
+    def past_states(
+        self, domain: Domain, for_only_ml_policy: bool = False
+    ) -> List[State]:
+        """Generate the past states of this tracker based on the history.
+
+        Args:
+            domain: a :class:`rasa.shared.core.domain.Domain`.
+            for_only_ml_policy: If True ignore dialogue turns that are present
+                only in rules.
+
+        Returns:
+            a list of states
+        """
         states_for_hashing = self.past_states_for_hashing(domain)
         return self._unfreeze_states(states_for_hashing)
 
diff --git a/rasa/shared/core/trackers.py b/rasa/shared/core/trackers.py
index 96988d3e7498..4b68c7d610c3 100644
--- a/rasa/shared/core/trackers.py
+++ b/rasa/shared/core/trackers.py
@@ -59,6 +59,7 @@
     SessionStarted,
     ActionExecutionRejected,
     EntitiesAdded,
+    HideRuleTurn,
 )
 from rasa.shared.core.domain import Domain, State
 from rasa.shared.core.slots import Slot
@@ -205,6 +206,9 @@ def __init__(
         self.latest_bot_utterance = None
         self._reset()
         self.active_loop: Dict[Text, Union[Text, bool, Dict, None]] = {}
+        self.hide_rule_turn = False
+        self.only_rule_slots = []
+        self.only_rule_loops = []
 
     ###
     # Public tracker interface
@@ -273,16 +277,20 @@ def freeze_current_state(state: State) -> FrozenState:
             }.items()
         )
 
-    def past_states(self, domain: Domain) -> List[State]:
+    def past_states(
+        self, domain: Domain, for_only_ml_policy: bool = False
+    ) -> List[State]:
         """Generate the past states of this tracker based on the history.
 
         Args:
-            domain: a :class:`rasa.shared.core.domain.Domain`
+            domain: a :class:`rasa.shared.core.domain.Domain`.
+            for_only_ml_policy: If True ignore dialogue turns that are present
+                only in rules.
 
         Returns:
             a list of states
         """
-        return domain.states_for_tracker_history(self)
+        return domain.states_for_tracker_history(self, for_only_ml_policy)
 
     def change_loop_to(self, loop_name: Optional[Text]) -> None:
         """Set the currently active loop.
diff --git a/tests/core/policies/test_rule_policy.py b/tests/core/policies/test_rule_policy.py
index 2add4ead31ac..bb3e770a1a27 100644
--- a/tests/core/policies/test_rule_policy.py
+++ b/tests/core/policies/test_rule_policy.py
@@ -20,7 +20,10 @@
     ACTION_BACK_NAME,
     RULE_SNIPPET_ACTION_NAME,
     REQUESTED_SLOT,
+    USER,
+    PREVIOUS_ACTION,
 )
+from rasa.shared.nlu.constants import TEXT, INTENT, ACTION_NAME
 from rasa.shared.core.domain import Domain
 from rasa.shared.core.events import (
     ActionExecuted,
@@ -29,6 +32,7 @@
     SlotSet,
     ActionExecutionRejected,
     LoopInterrupted,
+    HideRuleTurn,
 )
 from rasa.shared.nlu.interpreter import RegexInterpreter
 from rasa.core.nlg import TemplatedNaturalLanguageGenerator
@@ -1928,3 +1932,277 @@ def test_predict_nothing_if_fallback_disabled():
     )
 
     assert prediction.max_confidence == 0
+
+
+def test_hide_rule_turn():
+    chitchat = "chitchat"
+    action_chitchat = "action_chitchat"
+    domain = Domain.from_yaml(
+        f"""
+        intents:
+        - {GREET_INTENT_NAME}
+        - {chitchat}
+        actions:
+        - {UTTER_GREET_ACTION}
+        - {action_chitchat}
+    """
+    )
+    chitchat_story = TrackerWithCachedStates.from_events(
+        "chitchat story",
+        domain=domain,
+        slots=domain.slots,
+        evts=[
+            ActionExecuted(ACTION_LISTEN_NAME),
+            UserUttered(intent={"name": chitchat}),
+            ActionExecuted(action_chitchat),
+            ActionExecuted(ACTION_LISTEN_NAME),
+        ],
+    )
+    policy = RulePolicy()
+    policy.train(
+        [GREET_RULE, chitchat_story,], domain, RegexInterpreter(),
+    )
+
+    conversation_events = [
+        ActionExecuted(ACTION_LISTEN_NAME),
+        UserUttered("haha", {"name": GREET_INTENT_NAME}),
+    ]
+    prediction = policy.predict_action_probabilities(
+        DialogueStateTracker.from_events(
+            "casd", evts=conversation_events, slots=domain.slots
+        ),
+        domain,
+        RegexInterpreter(),
+    )
+    assert_predicted_action(prediction, domain, UTTER_GREET_ACTION)
+    assert isinstance(prediction.optional_events[0], HideRuleTurn)
+
+    conversation_events += prediction.optional_events
+    conversation_events += [
+        ActionExecuted(UTTER_GREET_ACTION),
+    ]
+    prediction = policy.predict_action_probabilities(
+        DialogueStateTracker.from_events(
+            "casd", evts=conversation_events, slots=domain.slots
+        ),
+        domain,
+        RegexInterpreter(),
+    )
+    assert_predicted_action(prediction, domain, ACTION_LISTEN_NAME)
+    assert isinstance(prediction.optional_events[0], HideRuleTurn)
+
+    conversation_events += prediction.optional_events
+    conversation_events += [
+        ActionExecuted(ACTION_LISTEN_NAME),
+        UserUttered("haha", {"name": chitchat}),
+    ]
+    tracker = DialogueStateTracker.from_events(
+        "casd", evts=conversation_events, slots=domain.slots
+    )
+    states = tracker.past_states(domain, for_only_ml_policy=True)
+    assert states == [
+        {},
+        {
+            USER: {TEXT: "haha", INTENT: chitchat},
+            PREVIOUS_ACTION: {ACTION_NAME: ACTION_LISTEN_NAME},
+        },
+    ]
+
+
+def test_hide_rule_turn_with_slots():
+    some_action = "some_action"
+    some_other_action = "some_other_action"
+    some_intent = "some_intent"
+    some_other_intent = "some_other_intent"
+    some_slot = "some_slot"
+    some_slot_value = "value1"
+    some_other_slot = "some_other_slot"
+    some_other_slot_value = "value2"
+    domain = Domain.from_yaml(
+        f"""
+    intents:
+    - {some_intent}
+    - {some_other_intent}
+    actions:
+    - {some_action}
+    - {some_other_action}
+    slots:
+      {some_slot}:
+        type: text
+      {some_other_slot}:
+        type: text
+        """
+    )
+
+    simple_rule = TrackerWithCachedStates.from_events(
+        "simple rule with an action that sets 1 slot",
+        domain=domain,
+        slots=domain.slots,
+        evts=[
+            ActionExecuted(RULE_SNIPPET_ACTION_NAME),
+            ActionExecuted(ACTION_LISTEN_NAME),
+            UserUttered(intent={"name": some_intent}),
+            ActionExecuted(some_action),
+            SlotSet(some_slot, some_slot_value),
+            ActionExecuted(ACTION_LISTEN_NAME),
+        ],
+        is_rule_tracker=True,
+    )
+    simple_rule_with_slot_set = TrackerWithCachedStates.from_events(
+        "simple rule with an additional slot set before it starts",
+        domain=domain,
+        slots=domain.slots,
+        evts=[
+            SlotSet(some_other_slot, some_other_slot_value),
+            ActionExecuted(RULE_SNIPPET_ACTION_NAME),
+            ActionExecuted(ACTION_LISTEN_NAME),
+            UserUttered(intent={"name": some_intent}),
+            ActionExecuted(some_action),
+            SlotSet(some_slot, some_slot_value),
+            ActionExecuted(ACTION_LISTEN_NAME),
+        ],
+        is_rule_tracker=True,
+    )
+    simple_story_with_other_slot_set = TrackerWithCachedStates.from_events(
+        "simple rule with an additional slot set before it starts",
+        domain=domain,
+        slots=domain.slots,
+        evts=[
+            ActionExecuted(ACTION_LISTEN_NAME),
+            UserUttered(intent={"name": some_other_action}),
+            ActionExecuted(some_other_action),
+            SlotSet(some_other_slot, some_other_slot_value),
+            ActionExecuted(ACTION_LISTEN_NAME),
+        ],
+    )
+
+    policy = RulePolicy()
+    policy.train(
+        [simple_rule, simple_rule_with_slot_set, simple_story_with_other_slot_set],
+        domain,
+        RegexInterpreter(),
+    )
+
+    conversation_events = [
+        ActionExecuted(ACTION_LISTEN_NAME),
+        UserUttered("haha", {"name": some_intent}),
+    ]
+    prediction = policy.predict_action_probabilities(
+        DialogueStateTracker.from_events(
+            "casd", evts=conversation_events, slots=domain.slots
+        ),
+        domain,
+        RegexInterpreter(),
+    )
+    assert_predicted_action(prediction, domain, some_action)
+    assert isinstance(prediction.optional_events[0], HideRuleTurn)
+    assert some_slot in prediction.optional_events[0].only_rule_slots
+    assert some_other_slot not in prediction.optional_events[0].only_rule_slots
+
+
+def test_hide_rule_turn_with_loops():
+    form_name = "some_form"
+    another_form_name = "another_form"
+    activate_form = "activate_form"
+    activate_another_form = "activate_another_form"
+    chitchat = "chitchat"
+    action_chitchat = "action_chitchat"
+    domain = Domain.from_yaml(
+        f"""
+        intents:
+        - {GREET_INTENT_NAME}
+        - {activate_form}
+        - {chitchat}
+        - {activate_another_form}
+        actions:
+        - {UTTER_GREET_ACTION}
+        - {action_chitchat}
+        slots:
+          {REQUESTED_SLOT}:
+            type: unfeaturized
+        forms:
+        - {form_name}
+        - {another_form_name}
+    """
+    )
+
+    form_activation_rule = _form_activation_rule(domain, form_name, activate_form)
+
+    another_form_activation_rule = _form_activation_rule(
+        domain, another_form_name, activate_another_form
+    )
+    another_form_activation_story = another_form_activation_rule.copy()
+    another_form_activation_story.is_rule_tracker = False
+
+    chitchat_story = TrackerWithCachedStates.from_events(
+        "chitchat story",
+        domain=domain,
+        slots=domain.slots,
+        evts=[
+            ActionExecuted(ACTION_LISTEN_NAME),
+            UserUttered(intent={"name": chitchat}),
+            ActionExecuted(action_chitchat),
+            ActionExecuted(ACTION_LISTEN_NAME),
+        ],
+    )
+    policy = RulePolicy()
+    policy.train(
+        [
+            form_activation_rule,
+            chitchat_story,
+            another_form_activation_rule,
+            another_form_activation_story,
+        ],
+        domain,
+        RegexInterpreter(),
+    )
+
+    conversation_events = [
+        ActionExecuted(ACTION_LISTEN_NAME),
+        UserUttered("haha", {"name": activate_form}),
+    ]
+    prediction = policy.predict_action_probabilities(
+        DialogueStateTracker.from_events(
+            "casd", evts=conversation_events, slots=domain.slots
+        ),
+        domain,
+        RegexInterpreter(),
+    )
+    assert_predicted_action(prediction, domain, form_name)
+    assert isinstance(prediction.optional_events[0], HideRuleTurn)
+    assert form_name in prediction.optional_events[0].only_rule_loops
+    assert another_form_name not in prediction.optional_events[0].only_rule_loops
+
+    conversation_events += prediction.optional_events
+    conversation_events += [
+        ActionExecuted(form_name),
+        ActiveLoop(form_name),
+    ]
+    prediction = policy.predict_action_probabilities(
+        DialogueStateTracker.from_events(
+            "casd", evts=conversation_events, slots=domain.slots
+        ),
+        domain,
+        RegexInterpreter(),
+    )
+    assert_predicted_action(prediction, domain, ACTION_LISTEN_NAME)
+    assert isinstance(prediction.optional_events[0], HideRuleTurn)
+    assert form_name in prediction.optional_events[0].only_rule_loops
+    assert another_form_name not in prediction.optional_events[0].only_rule_loops
+
+    conversation_events += prediction.optional_events
+    conversation_events += [
+        ActionExecuted(ACTION_LISTEN_NAME),
+        UserUttered("haha", {"name": chitchat}),
+    ]
+    tracker = DialogueStateTracker.from_events(
+        "casd", evts=conversation_events, slots=domain.slots
+    )
+    states = tracker.past_states(domain, for_only_ml_policy=True)
+    assert states == [
+        {},
+        {
+            USER: {TEXT: "haha", INTENT: chitchat},
+            PREVIOUS_ACTION: {ACTION_NAME: ACTION_LISTEN_NAME},
+        },
+    ]
diff --git a/tests/core/test_policies.py b/tests/core/test_policies.py
index 997eb5228352..71a10eafb433 100644
--- a/tests/core/test_policies.py
+++ b/tests/core/test_policies.py
@@ -793,9 +793,7 @@ def test_memorise_with_nlu(
 
         tracker = DialogueStateTracker(dialogue.name, default_domain.slots)
         tracker.recreate_from_dialogue(dialogue)
-        states = trained_policy.featurizer.prediction_states([tracker], default_domain)[
-            0
-        ]
+        states = trained_policy.prediction_states(tracker, default_domain)
 
         recalled = trained_policy.recall(states, tracker, default_domain)
         assert recalled is not None


Code review comments:
diff_hunk: @@ -621,11 +698,43 @@ def train(
             trackers_as_states, trackers_as_actions
         )
 
+    def train(
+        self,
+        training_trackers: List[TrackerWithCachedStates],
+        domain: Domain,
+        interpreter: NaturalLanguageInterpreter,
+        **kwargs: Any,
+    ) -> None:
+        """Trains the policy on given training trackers.
+
+        Args:
+            training_trackers: The list of the trackers.
+            domain: The domain.
+            interpreter: Interpreter which can be used by the polices for featurization.
+        """
+        # only consider original trackers (no augmented ones)
+        training_trackers = [
+            t
+            for t in training_trackers
+            if not hasattr(t, "is_augmented") or not t.is_augmented
path: rasa/core/policies/rule_policy.py
text: @user1:
Why is `or` needed here? Can it not be just `if not t.is_augmented`?

@author:
it is the difference between `TrackerWithCachedStates` and `DialogueStateTracker` you can actually pass both to train methods. I'd say refactoring it is out of scope of this PR