Repository: python-discord/bot

Issue:
Filter functionality for emoji-spam messages
Now that someone's figured out you can cause havok by dumping massive amounts of emojis into a channel, we need a filter to automatically manage this ASAP. Detection should be fairly simple, a very high emoji count is required for the effect (>20/message to >250 for varying impact) but emoji variance is not required. 

Tangentially: This is also likely related to a discord update that went out that has slowed the emoji loading process recently.

I think we can safely filter out and autodelete high emoji messages without significant false positives. 

We may want to handle both unicode emojis and server emojis. 

It's possible for unicode emojis to get a higher "load" because they could as one character for message transmission purposes but server emojis count for more. However, server emojis may be animated, and will apply significant load by that fact as well.




Discussion:
So what kinds of emojis does this affect?  Do standard unicode ones trigger it?  Is it specifically Discord or custom emoji?
FYI we already have [a rule for custom emojis](https://github.com/python-discord/bot/blob/master/bot/rules/discord_emojis.py), we can simply edit this one. 
I will fix it.
I will use `emoji` library (https://github.com/carpedm20/emoji/blob/master/emoji/unicode_codes.py) as Unicode emojis can be made from multiple emojis and this library takes care over parsing them as 1 emoji.

Resolving patch:
diff --git a/bot/rules/discord_emojis.py b/bot/rules/discord_emojis.py
--- a/bot/rules/discord_emojis.py
+++ b/bot/rules/discord_emojis.py
@@ -2,16 +2,17 @@
 from typing import Dict, Iterable, List, Optional, Tuple
 
 from discord import Member, Message
+from emoji import demojize
 
 
-DISCORD_EMOJI_RE = re.compile(r"<:\w+:\d+>")
+DISCORD_EMOJI_RE = re.compile(r"<:\w+:\d+>|:\w+:")
 CODE_BLOCK_RE = re.compile(r"```.*?```", flags=re.DOTALL)
 
 
 async def apply(
     last_message: Message, recent_messages: List[Message], config: Dict[str, int]
 ) -> Optional[Tuple[str, Iterable[Member], Iterable[Message]]]:
-    """Detects total Discord emojis (excluding Unicode emojis) exceeding the limit sent by a single user."""
+    """Detects total Discord emojis exceeding the limit sent by a single user."""
     relevant_messages = tuple(
         msg
         for msg in recent_messages
@@ -19,8 +20,9 @@ async def apply(
     )
 
     # Get rid of code blocks in the message before searching for emojis.
+    # Convert Unicode emojis to :emoji: format to get their count.
     total_emojis = sum(
-        len(DISCORD_EMOJI_RE.findall(CODE_BLOCK_RE.sub("", msg.content)))
+        len(DISCORD_EMOJI_RE.findall(demojize(CODE_BLOCK_RE.sub("", msg.content))))
         for msg in relevant_messages
     )
 


Test patch:
diff --git a/tests/bot/rules/test_discord_emojis.py b/tests/bot/rules/test_discord_emojis.py
--- a/tests/bot/rules/test_discord_emojis.py
+++ b/tests/bot/rules/test_discord_emojis.py
@@ -5,11 +5,12 @@
 from tests.helpers import MockMessage
 
 discord_emoji = "<:abcd:1234>"  # Discord emojis follow the format <:name:id>
+unicode_emoji = "🧪"
 
 
-def make_msg(author: str, n_emojis: int) -> MockMessage:
+def make_msg(author: str, n_emojis: int, emoji: str = discord_emoji) -> MockMessage:
     """Build a MockMessage instance with content containing `n_emojis` arbitrary emojis."""
-    return MockMessage(author=author, content=discord_emoji * n_emojis)
+    return MockMessage(author=author, content=emoji * n_emojis)
 
 
 class DiscordEmojisRuleTests(RuleTest):
@@ -20,16 +21,22 @@ def setUp(self):
         self.config = {"max": 2, "interval": 10}
 
     async def test_allows_messages_within_limit(self):
-        """Cases with a total amount of discord emojis within limit."""
+        """Cases with a total amount of discord and unicode emojis within limit."""
         cases = (
             [make_msg("bob", 2)],
             [make_msg("alice", 1), make_msg("bob", 2), make_msg("alice", 1)],
+            [make_msg("bob", 2, unicode_emoji)],
+            [
+                make_msg("alice", 1, unicode_emoji),
+                make_msg("bob", 2, unicode_emoji),
+                make_msg("alice", 1, unicode_emoji)
+            ],
         )
 
         await self.run_allowed(cases)
 
     async def test_disallows_messages_beyond_limit(self):
-        """Cases with more than the allowed amount of discord emojis."""
+        """Cases with more than the allowed amount of discord and unicode emojis."""
         cases = (
             DisallowedCase(
                 [make_msg("bob", 3)],
@@ -41,6 +48,20 @@ async def test_disallows_messages_beyond_limit(self):
                 ("alice",),
                 4,
             ),
+            DisallowedCase(
+                [make_msg("bob", 3, unicode_emoji)],
+                ("bob",),
+                3,
+            ),
+            DisallowedCase(
+                [
+                    make_msg("alice", 2, unicode_emoji),
+                    make_msg("bob", 2, unicode_emoji),
+                    make_msg("alice", 2, unicode_emoji)
+                ],
+                ("alice",),
+                4
+            )
         )
 
         await self.run_disallowed(cases)