Repository: RasaHQ/rasa
Pull request: Create metadata UserMessage attribute
Signed-off-by: ArthurTemporim <arthurrtl@gmail.com>

Fixes #3199 

**Proposed changes**:
- Add `metadata` attribute to UserMessage

**Status (please check what you already did)**:
- [x] made PR ready for code review
- [x] added some tests for the functionality
- [x] updated the documentation
- [x] updated the changelog
- [x] reformat files using `black` (please check [Readme](https://github.com/RasaHQ/rasa_nlu#code-style) for instructions)


Problem statement:
add metadata field to user message
We should a metadata field to the user message class so that you can add custom information (tokens, user info) to the user message in the input channel and use it in custom actions.

- [x] Add field in UserMessage class
- [x] Make sure it does not get dropped on the way to the action server
- [ ] Optional: Create some functions to get it on the action server

Hints from issue discussion:
Yes I am blocked right now because my external Api call in my custom action needs an access token for every user so this field may be a solution.

You can set an unfeaturized slot at the start by calling the core's event http endpoint (conversations/<sender_id>/tracker/events)
I think we should add this to the `UserUttered` event data as well (and make sure the `message_id` is also considered in the `as_dict` )
@arthurTemporim To make sure that the event actually goes to the action server you have to include in the `UserUttered` event data

Commit message: Improve docstring of UserMessage

Signed-off-by: ArthurTemporim <arthurrtl@gmail.com>

Patch to review:
diff --git a/CHANGELOG.rst b/CHANGELOG.rst
index 938e9ba516ef..1ea92afe410b 100644
--- a/CHANGELOG.rst
+++ b/CHANGELOG.rst
@@ -21,6 +21,7 @@ Changed
 -------
 - new event broker class: ``SQLProducer``. This event broker is now used when running locally with
   Rasa X
+- ``metadata`` attribute added to ``UserMessage``
 
 Removed
 -------
diff --git a/data/test_trackers/tracker_moodbot.json b/data/test_trackers/tracker_moodbot.json
index 939c417d6bc3..cf36fde2fe36 100644
--- a/data/test_trackers/tracker_moodbot.json
+++ b/data/test_trackers/tracker_moodbot.json
@@ -63,7 +63,7 @@
       },
       "event": "user",
       "text": "/greet",
-      "input_channel": null
+      "input_channel": null, "message_id": null, "metadata": null
     },
     {
       "timestamp": 1517821726.200373,
@@ -105,7 +105,7 @@
       },
       "event": "user",
       "text": "/mood_great",
-      "input_channel": null
+      "input_channel": null, "message_id": null, "metadata": null
     },
     {
       "timestamp": 1517821726.209908,
@@ -122,4 +122,4 @@
       "confidence": 1.0
     }
   ]
-}
\ No newline at end of file
+}
diff --git a/docs/user-guide/connectors/custom-connectors.rst b/docs/user-guide/connectors/custom-connectors.rst
index 7a1311a624a1..ea080bbc5b56 100644
--- a/docs/user-guide/connectors/custom-connectors.rst
+++ b/docs/user-guide/connectors/custom-connectors.rst
@@ -31,6 +31,28 @@ To send a message, you would run a command like:
 
 where ``myio`` is the name of your component.
 
+If you need to use extra information from your front end in your custom
+actions, you can add this information in the ``metadata`` dict of your user
+message. This information will accompany the user message through the rasa
+server into the action server when applicable, where you can find it stored in
+the ``tracker``. Message metadata will not directly affect NLU classification
+or action prediction.
+
+To send a message with metadata, you would run a command like:
+
+.. code-block:: bash
+
+    curl -XPOST http://localhost:5000/webhooks/myio/webhook \
+      -d '{"sender": "user1", "message": "here is my token: <token>", "metadata: {"type": "token"}}' \
+      -H "Content-type: application/json"
+
+Here are all the attributes of ``UserMessage``:
+
+.. autoclass:: rasa.core.channels.UserMessage
+
+   .. automethod:: __init__
+
+
 In your implementation of the ``receive`` endpoint, you need to make
 sure to call ``on_new_message(UserMessage(text, output, sender_id))``.
 This will tell Rasa Core to handle this user message. The ``output``
diff --git a/rasa/core/channels/channel.py b/rasa/core/channels/channel.py
index 8c31e96a5922..eba2f5566e6e 100644
--- a/rasa/core/channels/channel.py
+++ b/rasa/core/channels/channel.py
@@ -35,7 +35,20 @@ def __init__(
         parse_data: Dict[Text, Any] = None,
         input_channel: Optional[Text] = None,
         message_id: Optional[Text] = None,
+        metadata: Optional[Dict] = None,
     ) -> None:
+        """Creates a ``UserMessage`` object.
+
+        Args:
+            text: The message text content.
+            output_channel: The message output data.
+            sender_id: The message owner ID.
+            parse_data: Rasa data about the message.
+            input_channel: The message origin name.
+            message_id: ID of the message.
+            metadata: Additional metadata for this message.
+
+        """
         self.text = text.strip() if text else text
 
         if message_id is not None:
@@ -56,6 +69,7 @@ def __init__(
         self.input_channel = input_channel
 
         self.parse_data = parse_data
+        self.metadata = metadata
 
 
 def register(
@@ -395,7 +409,7 @@ async def on_message_wrapper(
         text: Text,
         queue: Queue,
         sender_id: Text,
-        input_channel,
+        input_channel: Text,
     ) -> None:
         collector = QueueOutputChannel(queue)
 
diff --git a/rasa/core/events/__init__.py b/rasa/core/events/__init__.py
index adbef099c2a3..c554bf4bf5e9 100644
--- a/rasa/core/events/__init__.py
+++ b/rasa/core/events/__init__.py
@@ -180,19 +180,21 @@ class UserUttered(Event):
 
     def __init__(
         self,
-        text,
+        text: Optional[Text] = None,
         intent=None,
         entities=None,
-        parse_data=None,
-        timestamp=None,
-        input_channel=None,
-        message_id=None,
+        parse_data: Optional[Dict[Text, Any]] = None,
+        timestamp: int = time.time(),
+        input_channel: Optional[Text] = None,
+        message_id: Optional[Text] = None,
+        metadata: Optional[Dict] = None,
     ):
         self.text = text
         self.intent = intent if intent else {}
         self.entities = entities if entities else []
         self.input_channel = input_channel
         self.message_id = message_id
+        self.metadata = metadata
 
         if parse_data:
             self.parse_data = parse_data
@@ -206,7 +208,9 @@ def __init__(
         super(UserUttered, self).__init__(timestamp)
 
     @staticmethod
-    def _from_parse_data(text, parse_data, timestamp=None, input_channel=None):
+    def _from_parse_data(
+        text, parse_data, timestamp=None, input_channel=None, metadata=None
+    ):
         return UserUttered(
             text,
             parse_data.get("intent"),
@@ -214,6 +218,7 @@ def _from_parse_data(text, parse_data, timestamp=None, input_channel=None):
             parse_data,
             timestamp,
             input_channel,
+            metadata,
         )
 
     def __hash__(self):
@@ -244,19 +249,18 @@ def __str__(self):
     def empty():
         return UserUttered(None)
 
-    def as_dict(self):
-        d = super(UserUttered, self).as_dict()
-        input_channel = None  # for backwards compatibility (persisted evemts)
-        if hasattr(self, "input_channel"):
-            input_channel = self.input_channel
-        d.update(
+    def as_dict(self) -> Dict[Text, Any]:
+        _dict = super(UserUttered, self).as_dict()
+        _dict.update(
             {
                 "text": self.text,
                 "parse_data": self.parse_data,
-                "input_channel": input_channel,
+                "input_channel": getattr(self, "input_channel", None),
+                "message_id": getattr(self, "message_id", None),
+                "metadata": getattr(self, "metadata", None),
             }
         )
-        return d
+        return _dict
 
     @classmethod
     def _from_story_string(cls, parameters: Dict[Text, Any]) -> Optional[List[Event]]:
@@ -267,6 +271,7 @@ def _from_story_string(cls, parameters: Dict[Text, Any]) -> Optional[List[Event]
                     parameters.get("parse_data"),
                     parameters.get("timestamp"),
                     parameters.get("input_channel"),
+                    parameters.get("metadata"),
                 )
             ]
         except KeyError as e:
diff --git a/rasa/core/processor.py b/rasa/core/processor.py
index 032f1d22f3e6..854b6ab3da97 100644
--- a/rasa/core/processor.py
+++ b/rasa/core/processor.py
@@ -311,6 +311,7 @@ async def _handle_message_with_tracker(
                 parse_data,
                 input_channel=message.input_channel,
                 message_id=message.message_id,
+                metadata=message.metadata,
             ),
             self.domain,
         )
diff --git a/tests/core/test_events.py b/tests/core/test_events.py
index 9210445bae09..0087cbf4ee06 100644
--- a/tests/core/test_events.py
+++ b/tests/core/test_events.py
@@ -78,6 +78,8 @@ def test_event_has_proper_implementation(one_event, another_event):
     "one_event",
     [
         UserUttered("/greet", {"name": "greet", "confidence": 1.0}, []),
+        UserUttered(metadata={"type": "text"}),
+        UserUttered(metadata=None),
         SlotSet("name", "rasa"),
         Restarted(),
         AllSlotsReset(),
@@ -135,6 +137,7 @@ def test_json_parse_user():
             },
             "entities": []
           },
+          "metadata": {},
         }
     # DOCS END
     # fmt: on
@@ -143,6 +146,7 @@ def test_json_parse_user():
         intent={"name": "greet", "confidence": 0.9},
         entities=[],
         parse_data={"intent": {"name": "greet", "confidence": 0.9}, "entities": []},
+        metadata={},
     )
 
 


Code review comments:
diff_hunk: @@ -206,14 +208,17 @@ def __init__(
         super(UserUttered, self).__init__(timestamp)
 
     @staticmethod
-    def _from_parse_data(text, parse_data, timestamp=None, input_channel=None):
+    def _from_parse_data(
+        text, parse_data, timestamp=None, input_channel=None, metadata=None
path: rasa/core/events/__init__.py
text: @user1:
would be great if you could add types

diff_hunk: @@ -206,14 +208,17 @@ def __init__(
         super(UserUttered, self).__init__(timestamp)
 
     @staticmethod
-    def _from_parse_data(text, parse_data, timestamp=None, input_channel=None):
+    def _from_parse_data(
+        text, parse_data, timestamp=None, input_channel=None, metadata=None
path: rasa/core/events/__init__.py
text: @user1:
can you please also handle the `message_id`

diff_hunk: @@ -35,7 +35,20 @@ def __init__(
         parse_data: Dict[Text, Any] = None,
         input_channel: Optional[Text] = None,
         message_id: Optional[Text] = None,
+        metadata: Optional[Dict] = None,
     ) -> None:
+        """Creates a ``UserMessage`` object.
+
+        Args:
+            text: The message text content.
+            output_channel: The message output data.
path: rasa/core/channels/channel.py
text: @user1:
```suggestion
            output_channel: the output channel which should be used to send bot responses back to the user.
```