Repository: opsdroid/opsdroid

Issue:
opsdroid slack connector intermittently ends up in an exception
<!-- Before you post an issue or if you are unsure about something join our matrix channel https://riot.im/app/#/room/#opsdroid-general:matrix.org and ask away! We are more than happy to help you. -->
# Description - opsdroid slack connector intermittently ends up in an exception    

this doesnt happen for all users - but i see that line 146 in   File "/usr/local/lib/python3.7/site-packages/opsdroid/connector/slack/__init__.py" is the culprit.

```
INFO opsdroid.connector.slack: Connected successfully.
INFO opsdroid.web: Started web server on http://0.0.0.0:8080
INFO opsdroid.core: Opsdroid is now running, press ctrl+c to exit.
DEBUG slack.rtm.client: The Websocket connection has been opened.
DEBUG opsdroid.parsers.crontab: Running crontab skills at Mon Feb 10 10:21:00 2020.
DEBUG slack.rtm.client: Running 1 callbacks for event: 'message'
DEBUG opsdroid.connector.slack: Looking up sender username.
ERROR slack.rtm.client: When calling '#process_message()' in the 'opsdroid.connector.slack' module the following error was raised: 'user'
DEBUG asyncio: Using selector: EpollSelector
Traceback (most recent call last):
  File "/usr/local/bin/opsdroid", line 8, in <module>
    sys.exit(cli())
  File "/usr/local/lib/python3.7/site-packages/click/core.py", line 764, in __call__
    return self.main(*args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/click/core.py", line 717, in main
    rv = self.invoke(ctx)
  File "/usr/local/lib/python3.7/site-packages/click/core.py", line 1137, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/local/lib/python3.7/site-packages/click/core.py", line 956, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/local/lib/python3.7/site-packages/click/core.py", line 555, in invoke
    return callback(*args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/opsdroid/cli/start.py", line 42, in start
    opsdroid.run()
  File "/usr/local/lib/python3.7/site-packages/opsdroid/core.py", line 165, in run
    self.eventloop.run_until_complete(asyncio.gather(*pending))
  File "/usr/local/lib/python3.7/asyncio/base_events.py", line 583, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.7/site-packages/slack/rtm/client.py", line 339, in _connect_and_read
    await self._read_messages()
  File "/usr/local/lib/python3.7/site-packages/slack/rtm/client.py", line 390, in _read_messages
    await self._dispatch_event(event, data=payload)
  File "/usr/local/lib/python3.7/site-packages/slack/rtm/client.py", line 437, in _dispatch_event
    rtm_client=self, web_client=self._web_client, data=data
  File "/usr/local/lib/python3.7/site-packages/opsdroid/connector/slack/__init__.py", line 146, in process_message
    user_info = await self.lookup_username(message["user"])
KeyError: 'user'
ERROR: Unhandled exception in opsdroid, exiting...
```

## Steps to Reproduce
Please also include relevant information and steps to reproduce the bug/issue.

i am not sure if this can be reproduced elsewhere - otherwise would have been reported by other users.
the slack channel has about 82 users.
the bot is part of 2 channels.
also users interact with the bot directly /



## Expected Functionality
no exception - Looking up sender username should succeed.

## Experienced Functionality
Explain what happened instead(Please include the debug log).

```INFO opsdroid.connector.slack: Connected successfully.
INFO opsdroid.web: Started web server on http://0.0.0.0:8080
INFO opsdroid.core: Opsdroid is now running, press ctrl+c to exit.
DEBUG slack.rtm.client: The Websocket connection has been opened.
DEBUG opsdroid.parsers.crontab: Running crontab skills at Mon Feb 10 10:21:00 2020.
DEBUG slack.rtm.client: Running 1 callbacks for event: 'message'
DEBUG opsdroid.connector.slack: Looking up sender username.
ERROR slack.rtm.client: When calling '#process_message()' in the 'opsdroid.connector.slack' module the following error was raised: 'user'
DEBUG asyncio: Using selector: EpollSelector
Traceback (most recent call last):
  File "/usr/local/bin/opsdroid", line 8, in <module>
    sys.exit(cli())
  File "/usr/local/lib/python3.7/site-packages/click/core.py", line 764, in __call__
    return self.main(*args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/click/core.py", line 717, in main
    rv = self.invoke(ctx)
  File "/usr/local/lib/python3.7/site-packages/click/core.py", line 1137, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/local/lib/python3.7/site-packages/click/core.py", line 956, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/local/lib/python3.7/site-packages/click/core.py", line 555, in invoke
    return callback(*args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/opsdroid/cli/start.py", line 42, in start
    opsdroid.run()
  File "/usr/local/lib/python3.7/site-packages/opsdroid/core.py", line 165, in run
    self.eventloop.run_until_complete(asyncio.gather(*pending))
  File "/usr/local/lib/python3.7/asyncio/base_events.py", line 583, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.7/site-packages/slack/rtm/client.py", line 339, in _connect_and_read
    await self._read_messages()
  File "/usr/local/lib/python3.7/site-packages/slack/rtm/client.py", line 390, in _read_messages
    await self._dispatch_event(event, data=payload)
  File "/usr/local/lib/python3.7/site-packages/slack/rtm/client.py", line 437, in _dispatch_event
    rtm_client=self, web_client=self._web_client, data=data
  File "/usr/local/lib/python3.7/site-packages/opsdroid/connector/slack/__init__.py", line 146, in process_message
    user_info = await self.lookup_username(message["user"])
KeyError: 'user'
ERROR: Unhandled exception in opsdroid, exiting...
```

## Versions
- **Opsdroid version:** latest master code.
- **Python version:** python3.7
- **OS/Docker version:** 18.06.3-ce




## Configuration File
Please include your version of the configuration file below.
configuration file passed in values.yaml helm chart

```yaml
  configuration: |
    welcome-message: true
    connectors:
      slack:
        token: "xxx"
        bot-name: "xxx" # default "opsdroid"
        default-room: "#xxx" # default "#general"
        #icon-emoji: ":smile:" # default ":robot_face:"
        connect-timeout: 10 # default 10 seconds
        chat-as-user: false # default false
    skills:
      - name: skill-yyy-statistics
        path: /home/skill/skill-yyy-statistics
        db_server: "1.1.1.1"
        db_name: "xx"
        user: "xxx"
        password: "xxx"
      - name: skill-yyy-help
        path: /home/skill/skill-yyy-help
      - name: skill-yyy-cache
        path: /home/skill/skill-yyy-cache
        db_server: "1.1.1.1"
        db_name: "zz"
        user: "xxx"
        password: "xxxx"
      - name: skill-yyy-eee
        path: /home/skill/skill-yyy-eee
      - name: skill-yyy-ttt
        path: /home/skill/skill-yyy-ttt


```

## Additional Details
Any other details you wish to include such as screenshots, console messages, etc.


<!-- Love opsdroid? Please consider supporting our collective:
 +👉  https://opencollective.com/opsdroid/donate -->

opsdroid slack connector intermittently ends up in an exception
<!-- Before you post an issue or if you are unsure about something join our matrix channel https://riot.im/app/#/room/#opsdroid-general:matrix.org and ask away! We are more than happy to help you. -->
# Description - opsdroid slack connector intermittently ends up in an exception    

this doesnt happen for all users - but i see that line 146 in   File "/usr/local/lib/python3.7/site-packages/opsdroid/connector/slack/__init__.py" is the culprit.

```
INFO opsdroid.connector.slack: Connected successfully.
INFO opsdroid.web: Started web server on http://0.0.0.0:8080
INFO opsdroid.core: Opsdroid is now running, press ctrl+c to exit.
DEBUG slack.rtm.client: The Websocket connection has been opened.
DEBUG opsdroid.parsers.crontab: Running crontab skills at Mon Feb 10 10:21:00 2020.
DEBUG slack.rtm.client: Running 1 callbacks for event: 'message'
DEBUG opsdroid.connector.slack: Looking up sender username.
ERROR slack.rtm.client: When calling '#process_message()' in the 'opsdroid.connector.slack' module the following error was raised: 'user'
DEBUG asyncio: Using selector: EpollSelector
Traceback (most recent call last):
  File "/usr/local/bin/opsdroid", line 8, in <module>
    sys.exit(cli())
  File "/usr/local/lib/python3.7/site-packages/click/core.py", line 764, in __call__
    return self.main(*args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/click/core.py", line 717, in main
    rv = self.invoke(ctx)
  File "/usr/local/lib/python3.7/site-packages/click/core.py", line 1137, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/local/lib/python3.7/site-packages/click/core.py", line 956, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/local/lib/python3.7/site-packages/click/core.py", line 555, in invoke
    return callback(*args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/opsdroid/cli/start.py", line 42, in start
    opsdroid.run()
  File "/usr/local/lib/python3.7/site-packages/opsdroid/core.py", line 165, in run
    self.eventloop.run_until_complete(asyncio.gather(*pending))
  File "/usr/local/lib/python3.7/asyncio/base_events.py", line 583, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.7/site-packages/slack/rtm/client.py", line 339, in _connect_and_read
    await self._read_messages()
  File "/usr/local/lib/python3.7/site-packages/slack/rtm/client.py", line 390, in _read_messages
    await self._dispatch_event(event, data=payload)
  File "/usr/local/lib/python3.7/site-packages/slack/rtm/client.py", line 437, in _dispatch_event
    rtm_client=self, web_client=self._web_client, data=data
  File "/usr/local/lib/python3.7/site-packages/opsdroid/connector/slack/__init__.py", line 146, in process_message
    user_info = await self.lookup_username(message["user"])
KeyError: 'user'
ERROR: Unhandled exception in opsdroid, exiting...
```

## Steps to Reproduce
Please also include relevant information and steps to reproduce the bug/issue.

i am not sure if this can be reproduced elsewhere - otherwise would have been reported by other users.
the slack channel has about 82 users.
the bot is part of 2 channels.
also users interact with the bot directly /



## Expected Functionality
no exception - Looking up sender username should succeed.

## Experienced Functionality
Explain what happened instead(Please include the debug log).

```INFO opsdroid.connector.slack: Connected successfully.
INFO opsdroid.web: Started web server on http://0.0.0.0:8080
INFO opsdroid.core: Opsdroid is now running, press ctrl+c to exit.
DEBUG slack.rtm.client: The Websocket connection has been opened.
DEBUG opsdroid.parsers.crontab: Running crontab skills at Mon Feb 10 10:21:00 2020.
DEBUG slack.rtm.client: Running 1 callbacks for event: 'message'
DEBUG opsdroid.connector.slack: Looking up sender username.
ERROR slack.rtm.client: When calling '#process_message()' in the 'opsdroid.connector.slack' module the following error was raised: 'user'
DEBUG asyncio: Using selector: EpollSelector
Traceback (most recent call last):
  File "/usr/local/bin/opsdroid", line 8, in <module>
    sys.exit(cli())
  File "/usr/local/lib/python3.7/site-packages/click/core.py", line 764, in __call__
    return self.main(*args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/click/core.py", line 717, in main
    rv = self.invoke(ctx)
  File "/usr/local/lib/python3.7/site-packages/click/core.py", line 1137, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/local/lib/python3.7/site-packages/click/core.py", line 956, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/local/lib/python3.7/site-packages/click/core.py", line 555, in invoke
    return callback(*args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/opsdroid/cli/start.py", line 42, in start
    opsdroid.run()
  File "/usr/local/lib/python3.7/site-packages/opsdroid/core.py", line 165, in run
    self.eventloop.run_until_complete(asyncio.gather(*pending))
  File "/usr/local/lib/python3.7/asyncio/base_events.py", line 583, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.7/site-packages/slack/rtm/client.py", line 339, in _connect_and_read
    await self._read_messages()
  File "/usr/local/lib/python3.7/site-packages/slack/rtm/client.py", line 390, in _read_messages
    await self._dispatch_event(event, data=payload)
  File "/usr/local/lib/python3.7/site-packages/slack/rtm/client.py", line 437, in _dispatch_event
    rtm_client=self, web_client=self._web_client, data=data
  File "/usr/local/lib/python3.7/site-packages/opsdroid/connector/slack/__init__.py", line 146, in process_message
    user_info = await self.lookup_username(message["user"])
KeyError: 'user'
ERROR: Unhandled exception in opsdroid, exiting...
```

## Versions
- **Opsdroid version:** latest master code.
- **Python version:** python3.7
- **OS/Docker version:** 18.06.3-ce




## Configuration File
Please include your version of the configuration file below.
configuration file passed in values.yaml helm chart

```yaml
  configuration: |
    welcome-message: true
    connectors:
      slack:
        token: "xxx"
        bot-name: "xxx" # default "opsdroid"
        default-room: "#xxx" # default "#general"
        #icon-emoji: ":smile:" # default ":robot_face:"
        connect-timeout: 10 # default 10 seconds
        chat-as-user: false # default false
    skills:
      - name: skill-yyy-statistics
        path: /home/skill/skill-yyy-statistics
        db_server: "1.1.1.1"
        db_name: "xx"
        user: "xxx"
        password: "xxx"
      - name: skill-yyy-help
        path: /home/skill/skill-yyy-help
      - name: skill-yyy-cache
        path: /home/skill/skill-yyy-cache
        db_server: "1.1.1.1"
        db_name: "zz"
        user: "xxx"
        password: "xxxx"
      - name: skill-yyy-eee
        path: /home/skill/skill-yyy-eee
      - name: skill-yyy-ttt
        path: /home/skill/skill-yyy-ttt


```

## Additional Details
Any other details you wish to include such as screenshots, console messages, etc.


<!-- Love opsdroid? Please consider supporting our collective:
 +👉  https://opencollective.com/opsdroid/donate -->



Discussion:
Thanks for raising this!

It looks like opsdroid is getting a message which does not have a user field. This is a little strange as I woudl expect all messages to have a user, perhaps some system messages do not or something. 

We should catch this `KeyError`, skip parsing and log it. 

If we also catch `KeyError` [here](https://github.com/opsdroid/opsdroid/blob/master/opsdroid/connector/slack/__init__.py#L147) and then add a log line with `_LOGGER.error` before the return [here](https://github.com/opsdroid/opsdroid/blob/master/opsdroid/connector/slack/__init__.py#L148) I think that would do it.
Thanks for raising this!

It looks like opsdroid is getting a message which does not have a user field. This is a little strange as I woudl expect all messages to have a user, perhaps some system messages do not or something. 

We should catch this `KeyError`, skip parsing and log it. 

If we also catch `KeyError` [here](https://github.com/opsdroid/opsdroid/blob/master/opsdroid/connector/slack/__init__.py#L147) and then add a log line with `_LOGGER.error` before the return [here](https://github.com/opsdroid/opsdroid/blob/master/opsdroid/connector/slack/__init__.py#L148) I think that would do it.

Resolving patch:
diff --git a/opsdroid/connector/slack/__init__.py b/opsdroid/connector/slack/__init__.py
--- a/opsdroid/connector/slack/__init__.py
+++ b/opsdroid/connector/slack/__init__.py
@@ -144,7 +144,8 @@ async def process_message(self, **payload):
         _LOGGER.debug(_("Looking up sender username."))
         try:
             user_info = await self.lookup_username(message["user"])
-        except ValueError:
+        except (ValueError, KeyError) as error:
+            _LOGGER.error(_("Username lookup failed for %s."), error)
             return
 
         # Replace usernames in the message


Test patch:
diff --git a/tests/test_connector_slack.py b/tests/test_connector_slack.py
--- a/tests/test_connector_slack.py
+++ b/tests/test_connector_slack.py
@@ -148,6 +148,11 @@ async def test_process_message(self):
         await connector.process_message(data=message)
         self.assertFalse(connector.opsdroid.parse.called)
 
+        connector.opsdroid.parse.reset_mock()
+        connector.lookup_username.side_effect = KeyError
+        await connector.process_message(data=message)
+        self.assertFalse(connector.opsdroid.parse.called)
+
     async def test_lookup_username(self):
         """Test that looking up a username works and that it caches."""
         connector = ConnectorSlack({"token": "abc123"}, opsdroid=OpsDroid())