internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: zulip/zulip

Issue:
scrollToHash can raise an exception if the browser does not support it
If the browser is Safari (or Google's webcrawler) and the page was loaded with an anchor of the form `#:~:text=something`, then calling `scrollToHash` will raise `Error: Syntax error, unrecognized expression` here:

https://github.com/zulip/zulip/blob/1676d0b638d91680badc474190a08a52c8dc7a36/web/src/portico/help.js#L112

We should catch it and do nothing, in that case.


Discussion:

Resolving patch:
diff --git a/zerver/views/documentation.py b/zerver/views/documentation.py
--- a/zerver/views/documentation.py
+++ b/zerver/views/documentation.py
@@ -9,6 +9,9 @@
 from django.http import HttpRequest, HttpResponse, HttpResponseNotFound
 from django.template import loader
 from django.views.generic import TemplateView
+from lxml import html
+from lxml.etree import Element, SubElement, XPath, _Element
+from markupsafe import Markup
 
 from zerver.context_processors import zulip_default_context
 from zerver.decorator import add_google_analytics_context
@@ -65,6 +68,9 @@ def get_context_data(self, **kwargs: Any) -> Dict[str, str]:
         return context
 
 
+sidebar_links = XPath("//a[@href=$url]")
+
+
 class MarkdownDirectoryView(ApiURLView):
     path_template = ""
     policies_view = False
@@ -215,7 +221,6 @@ def get_context_data(self, **kwargs: Any) -> Dict[str, Any]:
             )
             context["PAGE_DESCRIPTION"] = request_notes.placeholder_open_graph_description
 
-        context["sidebar_index"] = sidebar_index
         # An "article" might require the api_url_context to be rendered
         api_url_context: Dict[str, Any] = {}
         add_api_url_context(api_url_context, self.request)
@@ -223,6 +228,28 @@ def get_context_data(self, **kwargs: Any) -> Dict[str, Any]:
         context["api_url_context"] = api_url_context
         if endpoint_name and endpoint_method:
             context["api_url_context"]["API_ENDPOINT_NAME"] = endpoint_name + ":" + endpoint_method
+
+        sidebar_html = render_markdown_path(sidebar_index)
+        tree = html.fragment_fromstring(sidebar_html, create_parent=True)
+        if not context.get("page_is_policy_center", False):
+            home_h1 = Element("h1")
+            home_link = SubElement(home_h1, "a")
+            home_link.attrib["class"] = "no-underline"
+            home_link.attrib["href"] = context["doc_root"]
+            home_link.text = context["doc_root_title"] + " home"
+            tree.insert(0, home_h1)
+        url = context["doc_root"] + article
+        # Highlight current article link
+        links = sidebar_links(tree, url=url)
+        assert isinstance(links, list)
+        for a in links:
+            assert isinstance(a, _Element)
+            old_class = a.attrib.get("class", "")
+            assert isinstance(old_class, str)
+            a.attrib["class"] = old_class + " highlighted"
+        sidebar_html = "".join(html.tostring(child, encoding="unicode") for child in tree)
+        context["sidebar_html"] = Markup(sidebar_html)
+
         add_google_analytics_context(context)
         return context
 


Test patch:
Repository: zulip/zulip

Issue:
Cannot react with a custom emoji to a bot message 
Attempting to react to a bot message with a custom emoji appears to work but if you refresh the page it's gone. Inspecting the network requests reveals that the request to add the reaction fails with a 400 Bad Request error `Reaction doesn't exist.`.

This can be easily reproduced with the Notification Bot. Note that the `zulip` reaction works since it's not really a custom emoji, but you can reproduce the problem with any other custom emoji.


Discussion:
Hello @zulip/server-emoji members, this issue was labeled with the "area: emoji" label, so you may want to check it out!

<!-- areaLabelAddition -->
I think the key detail here is not that it's a bot message, but that it's a Notification bot message (i.e. we're dealing with a cross-realm user).  We're incorrectly using `message.sender.realm` here rather than `user_profile.realm`, I think:

```
        check_emoji_request(message.sender.realm, emoji_name,                                        
                            emoji_code, reaction_type)                                               
```

Longer-term, we will probably want to denormalize `Message.realm` and use that for queries like this.

@mateuszmandera would you be interested in working on that denormalization effort?  I think it could work very similarly to the Recipient project you did a couple months back.
Tagging as a priority since this is a correctness issue.  
@timabbott So should a temporary fix be filed or wait for `message.realm` to be denormalized?
There's a quick temporary fix we can do involving `user_profile.realm`, as mentioned above.  I imagine that should close this issue and we'll make a separate one for denormalization of `realm`.

Resolving patch:
diff --git a/zerver/views/reactions.py b/zerver/views/reactions.py
--- a/zerver/views/reactions.py
+++ b/zerver/views/reactions.py
@@ -73,7 +73,7 @@ def add_reaction(request: HttpRequest, user_profile: UserProfile, message_id: in
         # Otherwise, use the name provided in this request, but verify
         # it is valid in the user's realm (e.g. not a deactivated
         # realm emoji).
-        check_emoji_request(message.sender.realm, emoji_name,
+        check_emoji_request(user_profile.realm, emoji_name,
                             emoji_code, reaction_type)
 
     if user_message is None:


Test patch:
Repository: zulip/zulip

Issue:
Use 😛 as :P emoticon translation
#15590 added the (optional) translation of `:P` to `:stuck_out_tongue:`. However, Zulip renders `:stuck_out_tongue:` as [`😝 U+1F61D`](https://emojipedia.org/squinting-face-with-tongue/), while I would expect `:P`/`:p` to render as [`😛 U+1F61B`](https://emojipedia.org/face-with-tongue/) (called `:mischievous:` in Zulip). `U+1F61B` is what gets rendered for `:P`/`:p` on any other platform I am on (e.g. Slack, FB, Hangouts).

I guess https://github.com/zulip/zulip/commit/32d234ef8ae7421e1c0f1ef7e9b36bf1026967f0 would be a way to fix this, but actually maybe the emoji names for these ones should be changed. I find it strange that `😛 U+1F61B` would be called `:mischievous:`, and I think a more conventional name for it would be `:stuck_out_tongue:`.


Discussion:
Hello @zulip/server-emoji members, this issue was labeled with the "area: emoji" label, so you may want to check it out!

<!-- areaLabelAddition -->

Resolving patch:
diff --git a/tools/setup/emoji/emoji_names.py b/tools/setup/emoji/emoji_names.py
--- a/tools/setup/emoji/emoji_names.py
+++ b/tools/setup/emoji/emoji_names.py
@@ -34,11 +34,9 @@
     '1f60b': {'canonical_name': 'yum', 'aliases': []},
     # crazy from https://beebom.com/emoji-meanings/, seems like best emoji for
     # joking
+    '1f61b': {'canonical_name': 'stuck_out_tongue', 'aliases': ['mischievous']},
     '1f61c': {'canonical_name': 'stuck_out_tongue_wink', 'aliases': ['joking', 'crazy']},
-    '1f61d': {'canonical_name': 'stuck_out_tongue', 'aliases': []},
-    # don't really need two stuck_out_tongues (see People/23), so chose
-    # something else that could fit
-    '1f61b': {'canonical_name': 'mischievous', 'aliases': []},
+    '1f61d': {'canonical_name': 'stuck_out_tongue_closed_eyes', 'aliases': []},
     # kaching suggested by user
     '1f911': {'canonical_name': 'money_face', 'aliases': ['kaching']},
     # arms_open seems like a natural addition
diff --git a/version.py b/version.py
--- a/version.py
+++ b/version.py
@@ -44,4 +44,4 @@
 #   historical commits sharing the same major version, in which case a
 #   minor version bump suffices.
 
-PROVISION_VERSION = '93.0'
+PROVISION_VERSION = '94.0'


Test patch:
Repository: zulip/zulip

Issue:
Enable translations for hotspots subsystem
There are unused translations at the hotspots subsystem, which could be enabled due to finished and available translations. At the moment there is a mix of English and the configured user language.

Affected file: zerver/lib/hotspots.py

Example (mixed English/German):
![grafik](https://user-images.githubusercontent.com/7988080/91609700-28518580-e978-11ea-8073-c8f368253595.png)



Discussion:
Hello @zulip/server-i18n members, this issue was labeled with the "area: i18n" label, so you may want to check it out!

<!-- areaLabelAddition -->
Hmm, not sure what the bug is here; @vinitS101 @hackerkid can you investigate?
The "Hab's verstanden" (English: "Got it!") is translated at locale/[language]/translations.json, the rest of the text is translated at locale/[language]/LC_MESSAGES/django.po. Maybe that's the problem - sorry, I cannot investigate this any further :-)

Resolving patch:
diff --git a/zerver/lib/hotspots.py b/zerver/lib/hotspots.py
--- a/zerver/lib/hotspots.py
+++ b/zerver/lib/hotspots.py
@@ -3,7 +3,7 @@
 from typing import Dict, List
 
 from django.conf import settings
-from django.utils.translation import ugettext as _
+from django.utils.translation import ugettext_lazy as _
 
 from zerver.models import UserHotspot, UserProfile
 
@@ -44,8 +44,8 @@ def get_next_hotspots(user: UserProfile) -> List[Dict[str, object]]:
     if settings.ALWAYS_SEND_ALL_HOTSPOTS:
         return [{
             'name': hotspot,
-            'title': ALL_HOTSPOTS[hotspot]['title'],
-            'description': ALL_HOTSPOTS[hotspot]['description'],
+            'title': str(ALL_HOTSPOTS[hotspot]['title']),
+            'description': str(ALL_HOTSPOTS[hotspot]['description']),
             'delay': 0,
         } for hotspot in ALL_HOTSPOTS]
 
@@ -57,8 +57,8 @@ def get_next_hotspots(user: UserProfile) -> List[Dict[str, object]]:
         if hotspot not in seen_hotspots:
             return [{
                 'name': hotspot,
-                'title': ALL_HOTSPOTS[hotspot]['title'],
-                'description': ALL_HOTSPOTS[hotspot]['description'],
+                'title': str(ALL_HOTSPOTS[hotspot]['title']),
+                'description': str(ALL_HOTSPOTS[hotspot]['description']),
                 'delay': 0.5,
             }]
 


Test patch:
Repository: zulip/zulip

Issue:
New line character issue when using create_user management command 
The create_user management command reads password from a text file created by the server admin. To run this command I tried creating this text file using VIM, nano and echo (` echo pass > password.txt` without using `-n` flag). Each and every time new line character was automatically added to the end of the file. So if I set the content of file as `helloworld` and try to login to the server by entering `helloworld` it would not let me login since `\n` is missing. It was not obvious to me that the extra `\n` added by editors was the reason behind the server rejecting the credentials.

Should we remove the trailing `\n` character while reading the password from file?


Discussion:
@zulipbot label "area: production"

@timabbott Do you think we should have a label for management commands? I couldn't really find a good label for labelling this issue.
Hello @zulip/server-production members, this issue was labeled with the "area: production" label, so you may want to check it out!

<!-- areaLabelAddition -->
I've been using the `production` area label for these if production, or `tooling` if it's development-only.  I don't think there's enough issue volume for them to be worth their own label.

I agree, we should definitely `.strip()` the passwords in that command.

Resolving patch:
diff --git a/zerver/management/commands/create_user.py b/zerver/management/commands/create_user.py
--- a/zerver/management/commands/create_user.py
+++ b/zerver/management/commands/create_user.py
@@ -71,7 +71,7 @@ def handle(self, *args: Any, **options: Any) -> None:
         try:
             if options['password_file'] is not None:
                 with open(options['password_file']) as f:
-                    pw = f.read()
+                    pw = f.read().strip()
             elif options['password'] is not None:
                 pw = options['password']
             else:


Test patch:
Repository: zulip/zulip

Issue:
Convert favicons to be <canvas>.
In `./static/images/favicon` we have 100 different icons that were generated a while ago with the number of notifications that a user may have. Instead of calling a particular png every time the favicon is updated to reflect a new number of notifications, we should just generate a favicon with the <canvas> element.

This could be done with a combination of the green Zulip "z" icon and a text rendering with a shadow in the corner of the png. Then, we can kill all 100 favicons and not require any favicons to be loaded on update!

This could also be paired with upgrading favicons to be retina-supported as well.


Discussion:
Hello @zulip/server-refactoring members, this issue was labeled with the "area: refactoring" label, so you may want to check it out!

<!-- areaLabelAddition -->

Resolving patch:
diff --git a/tools/lib/template_parser.py b/tools/lib/template_parser.py
--- a/tools/lib/template_parser.py
+++ b/tools/lib/template_parser.py
@@ -303,6 +303,7 @@ def is_special_html_tag(s: str, tag: str) -> bool:
     'input',
     'path',
     'polygon',
+    'stop',
 ]
 
 def is_self_closing_html_tag(s: Text, tag: Text) -> bool:
diff --git a/version.py b/version.py
--- a/version.py
+++ b/version.py
@@ -44,4 +44,4 @@
 #   historical commits sharing the same major version, in which case a
 #   minor version bump suffices.
 
-PROVISION_VERSION = '112.1'
+PROVISION_VERSION = '112.2'


Test patch:
diff --git a/frontend_tests/node_tests/notifications.js b/frontend_tests/node_tests/notifications.js
--- a/frontend_tests/node_tests/notifications.js
+++ b/frontend_tests/node_tests/notifications.js
@@ -1,5 +1,7 @@
 "use strict";
 
+const rewiremock = require("rewiremock/node");
+
 // Dependencies
 set_global(
     "$",
@@ -32,7 +34,9 @@ zrequire("ui");
 zrequire("spoilers");
 spoilers.hide_spoilers_in_notification = () => {};
 
-zrequire("notifications");
+rewiremock.proxy(() => zrequire("notifications"), {
+    "../../static/js/favicon": {},
+});
 
 // Not muted streams
 const general = {
diff --git a/frontend_tests/node_tests/ui_init.js b/frontend_tests/node_tests/ui_init.js
--- a/frontend_tests/node_tests/ui_init.js
+++ b/frontend_tests/node_tests/ui_init.js
@@ -104,7 +104,9 @@ zrequire("narrow");
 zrequire("search_suggestion");
 zrequire("search");
 zrequire("tutorial");
-zrequire("notifications");
+rewiremock.proxy(() => zrequire("notifications"), {
+    "../../static/js/favicon": {},
+});
 zrequire("pm_conversations");
 zrequire("pm_list");
 zrequire("list_cursor");
Repository: zulip/zulip

Issue:
Translation for zh_TW gets ignored in some places
In the webapp, if I try switching to the translation for "Chinese (Taiwan)", a lot of the text on the screen is still untranslated:
![image](https://user-images.githubusercontent.com/28173/96927079-63010700-146b-11eb-843e-ebede1e91469.png)
That's even though many (at least) of those strings [do have translations in Transifex](https://www.transifex.com/zulip/zulip/translate/#zh_TW/$/67194598?q=text%3Asettings). Those translations have been there for months and do indeed seem to be in the repo, so it's not an issue of not having synced them.

I have a suspicion that the issue is with this code in `zerver/lib/i18n.py`:
```py3
def get_language_translation_data(language: str) -> Dict[str, str]:
    if language == 'en':
        return {}
    elif language == 'zh-hans':
        language = 'zh_Hans'
    elif language == 'zh-hant':
        language = 'zh_Hant'
    elif language == 'id-id':
        language = 'id_ID'
    path = os.path.join(settings.DEPLOY_ROOT, 'locale', language, 'translations.json')
    # …
```

That has a handful of special cases to try to translate between two different conventions for locale names. It sure looks like it'd need another one to support `zh_TW` aka `zh-tw`, and that without that this function will fail to do its job on zh_TW.

Better still, of course, would be to make this function stop being made of special cases. Here's a Django utility function that should do the job of all those cases: https://docs.djangoproject.com/en/2.2/ref/utils/#django.utils.translation.to_locale

---

~~Likely related, but possibly a separate issue: in the webapp language picker itself, the translation shouldn't be called "Chinese (Taiwan)" but rather something like "中文（台湾）" -- its name is written in English, whereas all the other languages have their names written in themselves.~~ (This other symptom is caused at least in part by #14565.)

(Both issues originally reported [in chat](https://chat.zulip.org/#narrow/stream/58-translation/topic/zh-Hant.20.2F.20zh_TW/near/1045033).)



Discussion:
Hello @zulip/server-i18n members, this issue was labeled with the "area: i18n" label, so you may want to check it out!

<!-- areaLabelAddition -->

Resolving patch:
diff --git a/zerver/lib/i18n.py b/zerver/lib/i18n.py
--- a/zerver/lib/i18n.py
+++ b/zerver/lib/i18n.py
@@ -73,13 +73,8 @@ def get_available_language_codes() -> List[str]:
 def get_language_translation_data(language: str) -> Dict[str, str]:
     if language == 'en':
         return {}
-    elif language == 'zh-hans':
-        language = 'zh_Hans'
-    elif language == 'zh-hant':
-        language = 'zh_Hant'
-    elif language == 'id-id':
-        language = 'id_ID'
-    path = os.path.join(settings.DEPLOY_ROOT, 'locale', language, 'translations.json')
+    locale = translation.to_locale(language)
+    path = os.path.join(settings.DEPLOY_ROOT, 'locale', locale, 'translations.json')
     try:
         with open(path, "rb") as reader:
             return orjson.loads(reader.read())


Test patch:
Repository: zulip/zulip

Issue:
Sentry integration has incorrect url when configured as a webhook
We have the sentry integration for zulip configured as a webhook, and the issue url in the notification is broken -- it has `event` in the path where it should have `events`.  I think the bug is here: https://github.com/zulip/zulip/blob/master/zerver/webhooks/sentry/view.py#L239

We're loving the new integration overall.  Thank you!


Discussion:

Resolving patch:
diff --git a/zerver/webhooks/sentry/view.py b/zerver/webhooks/sentry/view.py
--- a/zerver/webhooks/sentry/view.py
+++ b/zerver/webhooks/sentry/view.py
@@ -236,9 +236,10 @@ def transform_webhook_payload(payload: Dict[str, Any]) -> Optional[Dict[str, Any
     if not event_id:
         return None
 
-    event_path = f"event/{event_id}/"
+    event_path = f"events/{event_id}/"
     event['web_url'] = urljoin(payload['url'], event_path)
-    event['datetime'] = datetime.fromtimestamp(event['timestamp']).isoformat()
+    timestamp = event.get('timestamp', event['received'])
+    event['datetime'] = datetime.fromtimestamp(timestamp).isoformat()
     return payload
 
 


Test patch:
diff --git a/zerver/webhooks/sentry/tests.py b/zerver/webhooks/sentry/tests.py
--- a/zerver/webhooks/sentry/tests.py
+++ b/zerver/webhooks/sentry/tests.py
@@ -85,7 +85,7 @@ def test_event_for_exception_python(self) -> None:
     def test_webhook_event_for_exception_python(self) -> None:
         expected_topic = "ValueError: new sentry error."
         expected_message = """
-**New exception:** [ValueError: new sentry error.](https://sentry.io/organizations/bar-foundation/issues/1972208801/event/c916dccfd58e41dcabaebef0091f0736/)
+**New exception:** [ValueError: new sentry error.](https://sentry.io/organizations/bar-foundation/issues/1972208801/events/c916dccfd58e41dcabaebef0091f0736/)
 ```quote
 **level:** error
 **timestamp:** 2020-10-21 23:25:11
@@ -105,6 +105,17 @@ def test_webhook_event_for_exception_python(self) -> None:
 ```"""
         self.check_webhook("webhook_event_for_exception_python", expected_topic, expected_message)
 
+    def test_webhook_event_for_exception_javascript(self) -> None:
+        expected_topic = 'TypeError: can\'t access property "bar", x.foo is undefined'
+        expected_message = """
+**New exception:** [TypeError: can't access property "bar", x.foo is undefined](https://sentry.io/organizations/foo-bar-org/issues/1982047746/events/f3bf5fc4e354451db9e885a69b2a2b51/)
+```quote
+**level:** error
+**timestamp:** 2020-10-26 16:39:54
+**filename:** None
+```"""
+        self.check_webhook("webhook_event_for_exception_javascript", expected_topic, expected_message)
+
     def test_event_for_message_golang(self) -> None:
         expected_topic = "A test message event from golang."
         expected_message = """
Repository: zulip/zulip

Issue:
Document display settings in Zulip API documentation
This should look very similar to the existing documentation for the notification settings endpoint.  

When we do this, we should remember to update the `/register` documentation added in #16000 for display settings to include ir link to the detailed explanations of the meaning of these settings.


Discussion:
Hello @zulip/server-api members, this issue was labeled with the "area: documentation (api and integrations)" label, so you may want to check it out!

<!-- areaLabelAddition -->
I am working on it.
@timabbott I have added pr #17033 to address this, it would be great if you could review it.

Resolving patch:
diff --git a/zerver/openapi/python_examples.py b/zerver/openapi/python_examples.py
--- a/zerver/openapi/python_examples.py
+++ b/zerver/openapi/python_examples.py
@@ -1101,6 +1101,22 @@ def update_notification_settings(client: Client) -> None:
     validate_against_openapi_schema(result, "/settings/notifications", "patch", "200")
 
 
+@openapi_test_function("/settings/display:patch")
+def update_display_settings(client: Client) -> None:
+
+    # {code_example|start}
+    # Show user list on left sidebar in narrow windows.
+    # Change emoji set used for display to Google modern.
+    request = {
+        "left_side_userlist": True,
+        "emojiset": '"google"',
+    }
+    result = client.call_endpoint("settings/display", method="PATCH", request=request)
+    # {code_example|end}
+
+    validate_against_openapi_schema(result, "/settings/display", "patch", "200")
+
+
 @openapi_test_function("/user_uploads:post")
 def upload_file(client: Client) -> None:
     path_to_file = os.path.join(ZULIP_DIR, "zerver", "tests", "images", "img.jpg")
@@ -1391,6 +1407,7 @@ def test_users(client: Client, owner_client: Client) -> None:
     get_subscription_status(client)
     get_profile(client)
     update_notification_settings(client)
+    update_display_settings(client)
     upload_file(client)
     get_attachments(client)
     set_typing_status(client)


Test patch:
diff --git a/zerver/tests/test_openapi.py b/zerver/tests/test_openapi.py
--- a/zerver/tests/test_openapi.py
+++ b/zerver/tests/test_openapi.py
@@ -266,8 +266,6 @@ class OpenAPIArgumentsTest(ZulipTestCase):
         "/settings",
         "/users/me/avatar",
         "/users/me/api_key/regenerate",
-        # Not very useful outside the UI
-        "/settings/display",
         # Much more valuable would be an org admin bulk-upload feature.
         "/users/me/profile_data",
         #### Should be documented as part of interactive bots documentation