internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: zulip/zulip

Issue:
linting: Linter trips on "with" tags in Jinja.
Currently the code below fails with the error:

```
templates | Traceback (most recent call last):
templates |   File "tools/check-templates", line 168, in <module>
templates |     check_our_files(args.modified, args.all_dups, args.targets)
templates |   File "tools/check-templates", line 36, in check_our_files
templates |     check_html_templates(by_lang['html'], args.all_dups)
templates |   File "tools/check-templates", line 115, in check_html_templates
templates |     validate(fn=fn, check_indent=(fn not in bad_files))
templates |   File "/srv/zulip/tools/lib/template_parser.py", line 270, in validate
templates |     state.matcher(token)
templates |   File "/srv/zulip/tools/lib/template_parser.py", line 247, in f
templates |     ''' % (fn, problem, start_token.s, start_line, start_col, end_tag, end_line, end_col))
templates | lib.template_parser.TemplateParserException:
templates |                     fn: templates/zerver/features.html
templates |                     Mismatched tag.
templates |                     start:
templates |                         {% block portico_content %}
templates |                         line 11, col 1
templates |                     end tag:
templates |                         with
templates |                         line 15, col 5
templates | 
```

This seems to be because the linter doesn't understand the "with" and "endwith" tag being used in the templates. Here's the following code to trigger it:

```html
{% block portico_content %}

    {% with className="white" %}
        {% include 'zerver/landing_nav.html' %}
    {% endwith %}

{% endblock %}
```


Discussion:
Hello @zulip/server-tooling members, this issue was labeled with the **area: tooling** label, so you may want to check it out!
@adnrs96 is this easy to fix?
from the looks of it, feels like an easy fix. I am trying it out right now.

Resolving patch:
diff --git a/tools/lib/template_parser.py b/tools/lib/template_parser.py
--- a/tools/lib/template_parser.py
+++ b/tools/lib/template_parser.py
@@ -289,6 +289,7 @@ def is_django_block_tag(tag):
         'blocktrans',
         'trans',
         'raw',
+        'with',
     ]
 
 def get_handlebars_tag(text, i):


Test patch:
diff --git a/tools/tests/test_template_parser.py b/tools/tests/test_template_parser.py
--- a/tools/tests/test_template_parser.py
+++ b/tools/tests/test_template_parser.py
@@ -77,6 +77,15 @@ def test_validate_django(self):
             '''
         validate(text=my_html)
 
+        my_html = '''
+            {% block "content" %}
+                {% with className="class" %}
+                {% include 'foobar' %}
+                {% endwith %}
+            {% endblock %}
+            '''
+        validate(text=my_html)
+
     def test_validate_no_start_tag(self):
         # type: () -> None
         my_html = '''
Repository: zulip/zulip

Issue:
Remove unnecessary pathlib2 dependency
We have pathlib2 and scandir as dependencies just because of the code here to construct a `file://` URL:

`/tools/documentation_crawler/documentation_crawler/spiders/check_documentation.py`

(That's run as part of `tools/test-documentation`).  We should use some simpler mechanism in Python to do this and remove the unnecessary dependencies.


Discussion:
Hello @zulip/server-dependencies, @zulip/server-tooling members, this issue was labeled with the **area: dependencies**, **area: tooling** labels, so you may want to check it out!
@rht FYI, this might be a good follow-up to your recent work.
One particular advantage of fixing this is that `pathlib2` only depends on `scandir` for Python < 3.5.  Which means depending which Python version you run `update-locked-requirements` with, it may or may not be included as a listed dependency.  Since we support 3.4 and 3.5, we can defer dealing with the somewhat ugly work on having separate lock files for different Python versions if we just remove this.

Also worth noting that we'll eventually want separate lock files per-version, but it might be years before we actually need that, so I'd prefer to just remove the unnecessary dependency instead.

Resolving patch:
diff --git a/tools/documentation_crawler/documentation_crawler/spiders/check_documentation.py b/tools/documentation_crawler/documentation_crawler/spiders/check_documentation.py
--- a/tools/documentation_crawler/documentation_crawler/spiders/check_documentation.py
+++ b/tools/documentation_crawler/documentation_crawler/spiders/check_documentation.py
@@ -2,7 +2,7 @@
 from __future__ import print_function
 
 import os
-import pathlib2
+import pathlib
 
 from typing import List
 
@@ -16,7 +16,7 @@ def get_start_url():
     start_file = os.path.join(dir_path, os.path.join(*[os.pardir] * 4),
                               "docs/_build/html/index.html")
     return [
-        pathlib2.Path(os.path.abspath(start_file)).as_uri()
+        pathlib.Path(os.path.abspath(start_file)).as_uri()
     ]
 
 


Test patch:
Repository: zulip/zulip

Issue:
User mentions incorrectly considers deactivated users
If there's 2 users with the same name, one of them deactivated, mentioning can end up targeting the wrong user.

This seems to be a regression in `get_full_name_info`, which doesn't filter on `is_active=True`.

@showell can you take this one?  Seems like it should be quick to fix and test.


Discussion:
Hello @zulip/server-markdown members, this issue was labeled with the **area: markdown** label, so you may want to check it out!

Resolving patch:
diff --git a/zerver/lib/bugdown/__init__.py b/zerver/lib/bugdown/__init__.py
--- a/zerver/lib/bugdown/__init__.py
+++ b/zerver/lib/bugdown/__init__.py
@@ -1530,7 +1530,8 @@ def get_full_name_info(realm_id, full_names):
     }
 
     rows = UserProfile.objects.filter(
-        realm_id=realm_id
+        realm_id=realm_id,
+        is_active=True,
     ).filter(
         functools.reduce(lambda a, b: a | b, q_list),
     ).values(


Test patch:
diff --git a/zerver/tests/test_bugdown.py b/zerver/tests/test_bugdown.py
--- a/zerver/tests/test_bugdown.py
+++ b/zerver/tests/test_bugdown.py
@@ -10,6 +10,7 @@
 )
 from zerver.lib.alert_words import alert_words_in_realm
 from zerver.lib.camo import get_camo_url
+from zerver.lib.create_user import create_user
 from zerver.lib.emoji import get_emoji_url
 from zerver.lib.mention import possible_mentions
 from zerver.lib.message import render_markdown
@@ -34,6 +35,7 @@
     Realm,
     RealmFilter,
     Recipient,
+    UserProfile,
 )
 
 import copy
@@ -166,6 +168,39 @@ def bugdown_convert(text):
     # type: (Text) -> Text
     return bugdown.convert(text, message_realm=get_realm('zulip'))
 
+class BugdownMiscTest(ZulipTestCase):
+    def test_get_full_name_info(self):
+        # type: () -> None
+        realm = get_realm('zulip')
+
+        def make_user(email, full_name):
+            # type: (Text, Text) -> UserProfile
+            return create_user(
+                email=email,
+                password='whatever',
+                realm=realm,
+                full_name=full_name,
+                short_name='whatever',
+            )
+
+        fred1 = make_user('fred1@example.com', 'Fred Flintstone')
+        fred1.is_active = False
+        fred1.save()
+
+        fred2 = make_user('fred2@example.com', 'Fred Flintstone')
+
+        fred3 = make_user('fred3@example.com', 'Fred Flintstone')
+        fred3.is_active = False
+        fred3.save()
+
+        dct = bugdown.get_full_name_info(realm.id, {'Fred Flintstone', 'cordelia LEAR', 'Not A User'})
+        self.assertEqual(set(dct.keys()), {'fred flintstone', 'cordelia lear'})
+        self.assertEqual(dct['fred flintstone'], dict(
+            email='fred2@example.com',
+            full_name='Fred Flintstone',
+            id=fred2.id
+        ))
+
 class BugdownTest(ZulipTestCase):
     def load_bugdown_tests(self):
         # type: () -> Tuple[Dict[Text, Any], List[List[Text]]]
Repository: zulip/zulip

Issue:
Eliminate "Sending invitation" output spam in test_signup
We're getting close to eliminating output spam in our tests! One of the remaining cases is in `tools/test-backend zerver.tests.test_signup`, a bunch of lines like this:
```
2017-11-29 01:22:45.669 INFO [] Sending invitation for realm zulip to alice@zulip.com
```
We should work out where those are coming from and clean them up.



Discussion:
Hello @zulip/server-testing members, this issue was labeled with the **area: testing-coverage** label, so you may want to check it out!

<!-- areaLabelNotification -->
(See #1587 for locking this kind of problem down entirely in the future.)

@zulipbot claim

Resolving patch:
diff --git a/zerver/worker/queue_processors.py b/zerver/worker/queue_processors.py
--- a/zerver/worker/queue_processors.py
+++ b/zerver/worker/queue_processors.py
@@ -59,6 +59,7 @@
 import re
 import importlib
 
+logger = logging.getLogger(__name__)
 
 class WorkerDeclarationException(Exception):
     pass
@@ -232,7 +233,7 @@ def consume(self, data):
                 return
 
         referrer = get_user_profile_by_id(data["referrer_id"])
-        logging.info("Sending invitation for realm %s to %s" % (referrer.realm.string_id, invitee.email))
+        logger.info("Sending invitation for realm %s to %s" % (referrer.realm.string_id, invitee.email))
         do_send_confirmation_email(invitee, referrer)
 
         # queue invitation reminder for two days from now.


Test patch:
diff --git a/zproject/test_settings.py b/zproject/test_settings.py
--- a/zproject/test_settings.py
+++ b/zproject/test_settings.py
@@ -124,6 +124,7 @@ def set_loglevel(logger_name, level) -> None:
     set_loglevel('zulip.send_email', 'ERROR')
     set_loglevel('zerver.lib.digest', 'ERROR')
     set_loglevel('zerver.lib.email_mirror', 'ERROR')
+    set_loglevel('zerver.worker.queue_processors', 'WARNING')
 
 # Enable file:/// hyperlink support by default in tests
 ENABLE_FILE_LINKS = True
Repository: zulip/zulip

Issue:
Strikethrough doesn't work with auto-links
@tommyip pointed out that if you write something like `~~sdfsddsaf #7515 fffsdafdsa~~`, with an auto-linkified term inside strikethrough markup `~~`, the strikethrough doesn't happen:

![](https://chat.zulip.org/user_uploads/2/9b/jCZxws2flyppXlQWKDESvRf-/pasted_image.png)



Discussion:
Hello @zulip/server-markdown members, this issue was labeled with the **area: markdown** label, so you may want to check it out!

<!-- areaLabelNotification -->
@zulipbot claim.
Hello @akashnimare, you claimed this issue to work on it, but this issue and any referenced pull requests haven't been updated for 10 days. Are you still working on this issue?

If so, please update this issue by leaving a comment on this issue to let me know that you're still working on it. Otherwise, I'll automatically remove you from this issue in 4 days.

If you've decided to work on something else, simply comment `@zulipbot abandon` so that someone else can claim it and continue from where you left off.

Thank you for your valuable contributions to Zulip!

<!-- inactiveWarning -->

Resolving patch:
diff --git a/zerver/lib/bugdown/__init__.py b/zerver/lib/bugdown/__init__.py
--- a/zerver/lib/bugdown/__init__.py
+++ b/zerver/lib/bugdown/__init__.py
@@ -1389,7 +1389,7 @@ def extendMarkdown(self, md: markdown.Markdown, md_globals: Dict[str, Any]) -> N
         # Custom strikethrough syntax: ~~foo~~
         md.inlinePatterns.add('del',
                               markdown.inlinepatterns.SimpleTagPattern(
-                                  r'(?<!~)(\~\~)([^~{0}\n]+?)\2(?!~)', 'del'), '>strong')
+                                  r'(?<!~)(\~\~)([^~\n]+?)(\~\~)(?!~)', 'del'), '>strong')
 
         # Text inside ** must start and end with a word character
         # it need for things like "const char *x = (char *)y"


Test patch:
diff --git a/zerver/fixtures/markdown_test_cases.json b/zerver/fixtures/markdown_test_cases.json
--- a/zerver/fixtures/markdown_test_cases.json
+++ b/zerver/fixtures/markdown_test_cases.json
@@ -188,6 +188,12 @@
       "expected_output": "<p>I <del> like software </del> love hardware</p>",
       "text_content": "I  like software  love hardware"
     },
+    {
+      "name": "strikthrough_link",
+      "input": "~~test http://xx.xx link~~",
+      "expected_output": "<p><del>test <a href=\"http://xx.xx\" target=\"_blank\" title=\"http://xx.xx\">http://xx.xx</a> link</del></p>",
+      "text_content": "test http://xx.xx link"
+    },
     {
       "name": "underscore_disabled",
       "input": "_foo_",
Repository: zulip/zulip

Issue:
manual testing: Make it easy to subscribe to many streams.
We should add an option like `extra-streams` to `populate_db.py` that makes it easy to create lots of streams for manual testing.  We should subscribe Cordelia/Hamlet/etc. to most of the streams.
manual testing: Make it easy to subscribe to many streams.
We should add an option like `extra-streams` to `populate_db.py` that makes it easy to create lots of streams for manual testing.  We should subscribe Cordelia/Hamlet/etc. to most of the streams.


Discussion:
Hello @zulip/server-testing members, this issue was labeled with the **area: testing-infrastructure** label, so you may want to check it out!

<!-- areaLabelNotification -->
Hello @zulip/server-testing members, this issue was labeled with the **area: testing-infrastructure** label, so you may want to check it out!

<!-- areaLabelNotification -->

Resolving patch:
diff --git a/zilencer/management/commands/populate_db.py b/zilencer/management/commands/populate_db.py
--- a/zilencer/management/commands/populate_db.py
+++ b/zilencer/management/commands/populate_db.py
@@ -62,6 +62,12 @@ def add_arguments(self, parser: CommandParser) -> None:
                             default=0,
                             help='The number of extra bots to create')
 
+        parser.add_argument('--extra-streams',
+                            dest='extra_streams',
+                            type=int,
+                            default=0,
+                            help='The number of extra streams to create')
+
         parser.add_argument('--huddles',
                             dest='num_huddles',
                             type=int,
@@ -338,6 +344,20 @@ def handle(self, **options: Any) -> None:
                     "errors": {"description": "For errors", "invite_only": False},
                     "sales": {"description": "For sales discussion", "invite_only": False}
                 }  # type: Dict[Text, Dict[Text, Any]]
+
+                for i in range(options['extra_streams']):
+                    # Since a stream with name "Extra Stream 3" could show up
+                    # after "Extra Stream 29", pad the numbers with 0s.
+                    maximum_digits = len(str(options['extra_streams'] - 1))
+                    number_str = str(i).zfill(maximum_digits)
+
+                    extra_stream_name = 'Extra Stream ' + number_str
+
+                    zulip_stream_dict[extra_stream_name] = {
+                        "description": "Auto-generated extra stream.",
+                        "invite_only": False,
+                    }
+
                 bulk_create_streams(zulip_realm, zulip_stream_dict)
                 # Now that we've created the notifications stream, configure it properly.
                 zulip_realm.notifications_stream = get_stream("announce", zulip_realm)


Test patch:
Repository: zulip/zulip

Issue:
Errors when running `manage.py makemessages`
I get these errors in my local development environment:

```
(zulip-py3-venv) tabbott@zaset:~/zulip$ ./manage.py makemessages
UnicodeDecodeError: skipped file brainstorm_notes.txt in ./var/uploads/files/15/3d/47qkB-BgaArZ7wrTMTr-nsTK (reason: 'utf-8' codec can't decode byte 0x8b in position 1: invalid start byte)
UnicodeDecodeError: skipped file -.txt in ./var/uploads/files/15/9e/fqVojOZvoTZuGZ39r2_37NBn (reason: 'utf-8' codec can't decode byte 0x8b in position 1: invalid start byte)
UnicodeDecodeError: skipped file -.txt in ./var/uploads/files/2/fc/IfxNDeGaie57gWdOOok1Pyb5 (reason: 'utf-8' codec can't decode byte 0x8b in position 1: invalid start byte)
processing locale ca
processing locale es
```

I'm not sure why `manage.py makemessages` is parsing these uploaded files at all.   

@umairwaheed can you try to track this down?  We don't have a clear reproducer, but it seems like this sort of thing should be findable when reading code.


Discussion:
Hello @zulip/server-i18n members, this issue was labeled with the "area: i18n" label, so you may want to check it out!

<!-- areaLabelNotification -->
Sure. Will do on this weekend.

@zulipbot claim

Resolving patch:
diff --git a/zerver/management/commands/makemessages.py b/zerver/management/commands/makemessages.py
--- a/zerver/management/commands/makemessages.py
+++ b/zerver/management/commands/makemessages.py
@@ -133,6 +133,7 @@ def my_templatize(src: Text, *args: Any, **kwargs: Any) -> Text:
         try:
             ignore_patterns = options.get('ignore_patterns', [])
             ignore_patterns.append('docs/*')
+            ignore_patterns.append('var/*')
             options['ignore_patterns'] = ignore_patterns
             super().handle(*args, **options)
         finally:


Test patch:
Repository: zulip/zulip

Issue:
Import wildcard mentions from Slack into zulip
When a user does a wildcard mention (i.e. `@channel`, `@here`, etc.), we should translate those to a zulip wildcard mention.  I'd probably map them all to `@all` for now, but we should write the code in a way where changing the mapping is easy
Import wildcard mentions from Slack into zulip
When a user does a wildcard mention (i.e. `@channel`, `@here`, etc.), we should translate those to a zulip wildcard mention.  I'd probably map them all to `@all` for now, but we should write the code in a way where changing the mapping is easy


Discussion:
Already implemented for `@everyone`: https://github.com/zulip/zulip/blob/65c4a43a82c3e8b377e0d586671590a0fdec3a41/zerver/lib/slack_message_conversion.py#L77. Can be readily extended to other wildcards.
List of all the announcement types: `here`, `channel` and `everyone`. `all` has been deprecated. `group` is usually auto-completed to `channel` or list of users.

Ref:
1. https://get.slack.help/hc/en-us/articles/202009646-Make-an-announcement
Already implemented for `@everyone`: https://github.com/zulip/zulip/blob/65c4a43a82c3e8b377e0d586671590a0fdec3a41/zerver/lib/slack_message_conversion.py#L77. Can be readily extended to other wildcards.
List of all the announcement types: `here`, `channel` and `everyone`. `all` has been deprecated. `group` is usually auto-completed to `channel` or list of users.

Ref:
1. https://get.slack.help/hc/en-us/articles/202009646-Make-an-announcement

Resolving patch:
diff --git a/zerver/lib/slack_message_conversion.py b/zerver/lib/slack_message_conversion.py
--- a/zerver/lib/slack_message_conversion.py
+++ b/zerver/lib/slack_message_conversion.py
@@ -73,8 +73,12 @@ def convert_to_zulip_markdown(text: str, users: List[ZerverFieldsT],
     text = convert_markdown_syntax(text, SLACK_ITALIC_REGEX, "*")
 
     # Map Slack's mention all: '<!everyone>' to '@**all** '
+    # Map Slack's mention all: '<!channel>' to '@**all** '
+    # Map Slack's mention all: '<!here>' to '@**all** '
     # No regex for this as it can be present anywhere in the sentence
     text = text.replace('<!everyone>', '@**all**')
+    text = text.replace('<!channel>', '@**all**')
+    text = text.replace('<!here>', '@**all**')
 
     tokens = text.split(' ')
     for iterator in range(len(tokens)):


Test patch:
Repository: zulip/zulip

Issue:
Remove the "Delete streams" administrative tab
Now that we have a "Delete stream" button in the main streams UI, which has lots more context on description/traffic/subscribers, this page is useless.  We should just remove it.

We should make sure to remove the documentation in /help/ linking to this as well.


Discussion:
Hello @zulip/server-settings members, this issue was labeled with the "area: settings (admin/org)" label, so you may want to check it out!

<!-- areaLabelNotification -->
@zulipbot claim

Resolving patch:
diff --git a/zerver/lib/bugdown/help_settings_links.py b/zerver/lib/bugdown/help_settings_links.py
--- a/zerver/lib/bugdown/help_settings_links.py
+++ b/zerver/lib/bugdown/help_settings_links.py
@@ -35,8 +35,6 @@
     'deactivated-users-admin': ['Manage organization', 'Deactivated users',
                                 '/#organization/deactivated-users-admin'],
     'bot-list-admin': ['Manage organization', 'Bots', '/#organization/bot-list-admin'],
-    'streams-list-admin': ['Manage organization', 'Delete streams',
-                           '/#organization/streams-list-admin'],
     'default-streams-list': ['Manage organization', 'Default streams',
                              '/#organization/default-streams-list'],
     'filter-settings': ['Manage organization', 'Filter settings',


Test patch:
diff --git a/frontend_tests/node_tests/templates.js b/frontend_tests/node_tests/templates.js
--- a/frontend_tests/node_tests/templates.js
+++ b/frontend_tests/node_tests/templates.js
@@ -329,8 +329,7 @@ function render(template_name, args) {
     };
     var html = render('admin_tab', args);
     var admin_features = ["admin_users_table", "admin_bots_table",
-                          "admin_streams_table", "admin_deactivated_users_table",
-                          "admin_invites_table"];
+                          "admin_deactivated_users_table", "admin_invites_table"];
     _.each(admin_features, function (admin_feature) {
         assert.notEqual($(html).find("#" + admin_feature).length, 0);
     });