internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: mdn/kuma

Issue:
504 errors in Admin console
**Summary**
Most, if not all, of the BCSignal detail endpoints like https://developer.mozilla.org/admin/wiki/bcsignal/1764/change/ return 504

**Steps To Reproduce (STR)**
Visit https://developer.mozilla.org/admin/wiki/bcsignal/1764/change/

**Actual behavior**
504 error

**Expected behavior**
Should show the content of that `BCSignal` model instance.



Discussion:
I can kinda reproduce this locally. 
I have 18 entries on http://mdn.localhost:8000/admin/wiki/bcsignal/ and clicking any of them takes a loooong time to load. 3-7 seconds. 
For example, according to django-querycount I get this on an extreme one:

```
web_1            | http://mdn.localhost:8000/admin/wiki/bcsignal/13/change/
web_1            | |------|-----------|----------|----------|----------|------------|
web_1            | | Type | Database  |   Reads  |  Writes  |  Totals  | Duplicates |
web_1            | |------|-----------|----------|----------|----------|------------|
web_1            | | RESP |  default  |    7     |    0     |    7     |     1      |
web_1            | |------|-----------|----------|----------|----------|------------|
web_1            | Total queries: 7 in 33.0516s
```
Here's why...
<img width="1297" alt="Screen Shot 2019-10-18 at 11 46 30 AM" src="https://user-images.githubusercontent.com/26739/67108685-0787a380-f19d-11e9-851b-09200fe26536.png">

It's selecting every single Wiki document. I think it's potentially related to the filtering options not being defined so it creates filtering options automatically based on the struct of the ORM models. 
We just need to make this not a drop-down of every single possible document:
<img width="1460" alt="Screen Shot 2019-10-18 at 12 11 19 PM" src="https://user-images.githubusercontent.com/26739/67110324-774b5d80-f1a0-11e9-899d-8880462c58f4.png">


Resolving patch:
diff --git a/kuma/wiki/admin.py b/kuma/wiki/admin.py
--- a/kuma/wiki/admin.py
+++ b/kuma/wiki/admin.py
@@ -612,3 +612,14 @@ class EditorToolbarAdmin(admin.ModelAdmin):
 @admin.register(BCSignal)
 class BCSignalAdmin(admin.ModelAdmin):
     list_display = ['pk', 'document', 'submitted_at']
+    list_filter = ['submitted_at']
+    search_fields = ['document__slug', 'document__title', 'document__locale']
+    readonly_fields = ['document', 'browsers', 'feature']
+
+    def get_queryset(self, request):
+        only = (
+            'document__slug', 'document__title', 'document__locale',
+            'submitted_at'
+        )
+        return super(BCSignalAdmin, self).get_queryset(request).select_related(
+            'document').only(*only)


Test patch:
Repository: mdn/kuma

Issue:
Newsletter JS should load deferred
We're currently loading the newsletter.js bundle with `async`. There's probably no reason to do that. 
Thankfully the script tag for it is far down in the HTML so the chances are smaller that it'll cause harm. 

`async` script tags *execute* as soon as they're ready. Unlike `defer` which always execute after the rendering phase. This means that `async` script tags can cause the rendering to have to pause as it CPU processes the `async` script. 

One common scenario is if you click on a page, then click an internal link. This time, the script tag gets noticed in the HTML and the browser realizes that it has it in the browser cache, so it executes it immediately. This might mean it's blocking other important rendering and DOM event handlers. 


Discussion:
There are actually multiple script tags with `'extra_context': { 'async': True, },` in `kuma/settings/common.py`. Many that shouldn't be. From a quick look, many of them are unimportant since they're part of the wiki. But much of the Wiki relies on `window.jQuery` as a global so there might be race conditions lurking there. 
The only one I can see that should remain `async` is the `perf.js` because that one is so important that it's worth stopping the rendering for that. Perhaps I don't understand what `perf.js` but the name seems to imply that it wants to start doing its thing as soon as possible so it can start measuring stuff. 
Hmm.. I just occurred to me that the `newsletter.js` is only used on the home page(s). Performance might be unimportant on those. 

Resolving patch:
diff --git a/kuma/settings/common.py b/kuma/settings/common.py
--- a/kuma/settings/common.py
+++ b/kuma/settings/common.py
@@ -1088,7 +1088,7 @@ def pipeline_one_scss(slug, **kwargs):
         ),
         'output_filename': 'build/js/newsletter.js',
         'extra_context': {
-            'async': True,
+            'defer': True,
         },
     },
     'perf': {


Test patch:
Repository: mdn/kuma

Issue:
Redirect additional MDN pages to Extension Workshop 
Hi there! We need the following pages to be redirected from MDN to Extension Workshop: 

- https://developer.mozilla.org/docs/Mozilla/Add-ons/AMO/Blocking_Process should redirect to https://extensionworkshop.com/documentation/publish/add-ons-blocking-process/
- https://developer.mozilla.org/docs/Mozilla/Add-ons/Third_Party_Library_Usage should redirect to https://extensionworkshop.com/documentation/publish/third-party-library-usage/
- https://developer.mozilla.org/docs/Mozilla/Add-ons/WebExtensions/What_does_review_rejection_mean_to_users should redirect to https://extensionworkshop.com/documentation/publish/what-does-review-rejection-mean-to-users/

It would be great if this could happen by end of day October 25, but we have some flexibility. 




Discussion:
Hi @caitmuenster,

Is there any way that can you give us more heads up for these requests?

We're trying to do abide by (Agile) Sprints for planning our work (having things planned a week before we start the actual work), and support tickets are usually weighed against all the other often urgent items in our backlog. 

The other reason, is that we don't usually do deployments on Fridays (in case something goes sideways), and deployments are arduous for us right now (they usually can take a few hours), but we're working on improving that.

Thanks for your patience, 
Tobin
Hey @tobinmori, sorry about that! Yes, I can give more lead time for these requests in the future. :) Thanks for scheduling this request for an upcoming sprint! 

Resolving patch:
diff --git a/kuma/redirects/redirects.py b/kuma/redirects/redirects.py
--- a/kuma/redirects/redirects.py
+++ b/kuma/redirects/redirects.py
@@ -946,6 +946,12 @@ def locale_redirect(pattern, to, prepend_locale=True, **kwargs):
          'publish/distribute-for-desktop-apps/'),
         ('WebExtensions/Distribution_options/Add-ons_in_the_enterprise',
          'enterprise/'),
+        ('AMO/Blocking_Process',
+         'publish/add-ons-blocking-process/'),
+        ('Third_Party_Library_Usage',
+         'publish/third-party-library-usage/'),
+        ('WebExtensions/What_does_review_rejection_mean_to_users',
+         'publish/what-does-review-rejection-mean-to-users/'),
     )
 ]
 


Test patch:
diff --git a/tests/headless/map_301.py b/tests/headless/map_301.py
--- a/tests/headless/map_301.py
+++ b/tests/headless/map_301.py
@@ -557,5 +557,11 @@
          'publish/distribute-for-desktop-apps/'),
         ('WebExtensions/Distribution_options/Add-ons_in_the_enterprise',
          'enterprise/'),
+        ('AMO/Blocking_Process',
+         'publish/add-ons-blocking-process/'),
+        ('Third_Party_Library_Usage',
+         'publish/third-party-library-usage/'),
+        ('WebExtensions/What_does_review_rejection_mean_to_users',
+         'publish/what-does-review-rejection-mean-to-users/'),
     )
 ))
Repository: mdn/kuma

Issue:
TypeError: hrefAttrValue is null
**Summary**
`local-anchor.js` assumes that every `<a>` element in the DOM has a `href`. But in `wiki-compat-signal.js` there is a `<a>` without a href:
https://github.com/mdn/kuma/blob/fe06cc4abf710956d74df75491119f159f3f46ed/kuma/static/js/wiki-compat-signal.js#L14

I don't yet know when exactly it happens because I couldn't immediately reproduce it [in prod](https://developer.mozilla.org/en-US/docs/Web/HTML/Element/body#Browser_compatibility)




Discussion:
<img width="1500" alt="Screen Shot 2019-11-25 at 8 55 19 AM" src="https://user-images.githubusercontent.com/26739/69546419-b9b34780-0f61-11ea-8c3c-653ca4ca7a4a.png">

@atopal @Elchi3 Can we just delete the old BC signal thing? The one that's only loaded on wiki.developer.m.o. The one that doesn't display a pretty form with actual questions. 
sure
Actually, it happens every time. 
<img width="1616" alt="Screen Shot 2019-11-25 at 9 00 05 AM" src="https://user-images.githubusercontent.com/26739/69546590-0860e180-0f62-11ea-9b2b-a6bbcda70a2e.png">


Resolving patch:
diff --git a/kuma/settings/common.py b/kuma/settings/common.py
--- a/kuma/settings/common.py
+++ b/kuma/settings/common.py
@@ -1077,13 +1077,6 @@ def pipeline_one_scss(slug, **kwargs):
         'output_filename': 'build/js/wiki-compat-tables.js',
         'template_name': 'pipeline/javascript-array.jinja',
     },
-    'wiki-compat-signal': {
-        'source_filenames': (
-            'js/wiki-compat-signal.js',
-        ),
-        'output_filename': 'build/js/wiki-compat-signal.js',
-        'template_name': 'pipeline/javascript-array.jinja',
-    },
     'task-completion': {
         'source_filenames': (
             'js/task-completion.js',


Test patch:
Repository: mdn/kuma

Issue:
New front-end: No "partial localization"/"localization in progress" warning box
**Summary**
On w.d.m.o ("old" front-end), when a page has the "Localization in progress" checkbox ticked, a box is displayed at the top of the article.
This message varies whether the last edit is recent/fresh or not (in this case, one can safely guess the translation has been abandoned).

cf. https://github.com/mdn/kuma/blob/6c99355a6623474bca20abd7c18f4f22a47cdeaf/kuma/wiki/jinja2/wiki/document.html#L206-L214

This does not seem to be displayed on the new front-end, currently on d.m.o

URL to reproduce: https://wiki.developer.mozilla.org/fr/docs/Web/API/API_HTML_Drag_and_Drop/Opérations_de_glissement

**Rationale**
As some of the locales are not heavily maintained, a reader should have a way to be warned about the "staleness" of the current "localized" document they are reading.


**Audience**
Users consulting localized pages on MDN


**Proposal**
Users using the new front-end and navigating to unmaintained/partly translated pages should see the same box as they can see on w.d.m.o.
As a result, such users could be aware that the content may be incorrect and switch to a, hopefully, better maintained English page instead of being frustrated of seeing incorrect/stale content.


**Additional context**
Somehow related to https://github.com/mdn/kuma/issues/5806



Discussion:
![2019-10-25_08-25-26](https://user-images.githubusercontent.com/2413436/67548072-2dfb9080-f701-11e9-9469-85f8458b5c8d.png)


Resolving patch:
diff --git a/kuma/api/v1/views.py b/kuma/api/v1/views.py
--- a/kuma/api/v1/views.py
+++ b/kuma/api/v1/views.py
@@ -140,6 +140,10 @@ def document_api_data(doc=None, redirect_url=None):
         {doc.locale} | set(t.locale for t in other_translations))
 
     doc_absolute_url = doc.get_absolute_url()
+    revision = doc.current_or_latest_revision()
+    translation_status = None
+    if doc.parent_id and revision and revision.localization_in_progress:
+        translation_status = 'outdated' if revision.translation_age >= 10 else 'in-progress'
     return {
         'documentData': {
             'locale': doc.locale,
@@ -164,6 +168,7 @@ def document_api_data(doc=None, redirect_url=None):
                 if doc.is_localizable else
                 None
             ),
+            'translationStatus': translation_status,
             'bodyHTML': doc.get_body_html(),
             'quickLinksHTML': doc.get_quick_links_html(),
             'tocHTML': doc.get_toc_html(),


Test patch:
diff --git a/kuma/javascript/src/document.test.js b/kuma/javascript/src/document.test.js
--- a/kuma/javascript/src/document.test.js
+++ b/kuma/javascript/src/document.test.js
@@ -21,6 +21,7 @@ export const fakeDocumentData = {
     language: 'English (US)',
     hrefLang: 'en',
     absoluteURL: '[fake absolute url]',
+    translationStatus: null,
     translateURL: '[fake translate url]',
     wikiURL:
         'https://wiki.mdn.developer.mozilla.org/en-US/docs/Web/CSS/box-shadow',
Repository: mdn/kuma

Issue:
ClientError (Throttling) in request_cdn_cache_invalidation()
https://sentry.prod.mozaws.net/operations/mdn-prod/issues/6766908/

```
ClientError: An error occurred (Throttling) when calling the CreateInvalidation operation (reached max retries: 4): Rate exceeded
(1 additional frame(s) were not displayed)
...
  File "newrelic/hooks/application_celery.py", line 85, in wrapper
    return wrapped(*args, **kwargs)
  File "celery/app/trace.py", line 648, in __protected_call__
    return self.run(*args, **kwargs)
  File "kuma/api/tasks.py", line 180, in request_cdn_cache_invalidation
    invalidation = client.create_invalidation(
  File "botocore/client.py", line 357, in _api_call
    return self._make_api_call(operation_name, kwargs)
  File "botocore/client.py", line 661, in _make_api_call
    raise error_class(parsed_response, operation_name)

ClientError: An error occurred (Throttling) when calling the CreateInvalidation operation (reached max retries: 4): Rate exceeded
```


Discussion:

Resolving patch:
diff --git a/kuma/api/tasks.py b/kuma/api/tasks.py
--- a/kuma/api/tasks.py
+++ b/kuma/api/tasks.py
@@ -4,9 +4,11 @@
 import time
 
 import boto3
+from botocore.exceptions import ClientError
 from celery import task
 from django.conf import settings
 from django.utils.module_loading import import_string
+from redo import retriable
 
 from kuma.core.utils import chunked
 from kuma.wiki.models import Document
@@ -153,6 +155,7 @@ def publish(doc_pks, log=None, completion_message=None,
 
 
 @task
+@retriable(retry_exceptions=(ClientError,))
 def request_cdn_cache_invalidation(doc_locale_slug_pairs, log=None):
     """
     Trigger an attempt to purge the given documents from one or more


Test patch:
Repository: mdn/kuma

Issue:
BCD Signaling doesn't need to be a waffle flag
Spawned from: https://github.com/mdn/kuma/pull/6108#issuecomment-561241527

BCD Signalling is and will be on for everyone. 

For good measure we shouldn't just remove all waffle stuff but also move it to `settings/common.py`. 



Discussion:

Resolving patch:
diff --git a/kuma/api/v1/views.py b/kuma/api/v1/views.py
--- a/kuma/api/v1/views.py
+++ b/kuma/api/v1/views.py
@@ -9,7 +9,6 @@
 from rest_framework.decorators import api_view
 from rest_framework.renderers import JSONRenderer
 from rest_framework.response import Response
-from waffle.decorators import waffle_flag
 from waffle.models import Flag, Sample, Switch
 
 from kuma.api.v1.serializers import BCSignalSerializer
@@ -298,9 +297,11 @@ class APISearchView(SearchView):
 
 
 @ratelimit(key='user_or_ip', rate='10/d', block=True)
-@waffle_flag('bc-signals')
 @api_view(['POST'])
 def bc_signal(request):
+    if not settings.ENABLE_BCD_SIGNAL:
+        return Response("not enabled", status=status.HTTP_400_BAD_REQUEST)
+
     serializer = BCSignalSerializer(data=request.data)
     if serializer.is_valid():
         serializer.save()
diff --git a/kuma/core/migrations/0003_auto_20191212_0638.py b/kuma/core/migrations/0003_auto_20191212_0638.py
new file mode 100644
--- /dev/null
+++ b/kuma/core/migrations/0003_auto_20191212_0638.py
@@ -0,0 +1,24 @@
+# -*- coding: utf-8 -*-
+# Generated by Django 1.11.23 on 2019-12-12 06:38
+from __future__ import unicode_literals
+
+from django.db import migrations
+
+
+def remove_bcd_signal_waffle_flag(apps, schema_editor):
+    Flag = apps.get_model('waffle', 'Flag')
+    Flag.objects.filter(name='bc-signals').delete()
+
+
+class Migration(migrations.Migration):
+
+    dependencies = [
+        ('core', '0002_auto_20191206_0805'),
+        # This is needed otherwise `apps.get_model('waffle', 'Flag')`
+        # will raise a Django app LookupError.
+        ('waffle', '0001_initial'),
+    ]
+
+    operations = [
+        migrations.RunPython(remove_bcd_signal_waffle_flag)
+    ]
diff --git a/kuma/settings/common.py b/kuma/settings/common.py
--- a/kuma/settings/common.py
+++ b/kuma/settings/common.py
@@ -1822,6 +1822,10 @@ def get_user_url(user):
 # Newsletter Signup Settings
 ENABLE_NEWSLETTER_SIGNUP = config('ENABLE_NEWSLETTER_SIGNUP', default=False, cast=bool)
 
+# Whether or not to enable the BCD signalling feature.
+# Affects loading of CSS (statically) and JS (in runtime).
+ENABLE_BCD_SIGNAL = config('ENABLE_BCD_SIGNAL', default=True, cast=bool)
+
 # Content Experiments
 # Must be kept up to date with PIPELINE_JS setting and the JS client-side
 #  configuration. The 'id' should be a key in PIPELINE_JS, that loads


Test patch:
Repository: mdn/kuma

Issue:
Move developer_needs to waffle Switch
Spawned from: https://github.com/mdn/kuma/pull/6108#issuecomment-561241527

We're currently doing this:
```
  {% if waffle.flag('developer_needs') %}
    {% stylesheet 'banners' %}
  {% endif %}
```
...to determine loading the banners CSS. 

It doesn't need to be a waffle FLAG and it can't be either because a Flag doesn't work for CSS. It only works for the `/api/v1/whoami` which triggers the loading of `banners.jsx` in runtime. 

It should become this:
```diff
- {% if waffle.flag('developer_needs') %}
+ {% if waffle.switch('developer_needs') %}
    {% stylesheet 'banners' %}
  {% endif %}
```

Also, *today* the only factor that determines if you need the banners CSS is `developer_needs` but it can become more than that. So it might one day evolve to:

```
 {% if waffle.switch('developer_needs') or waffle.switch('other_thing') %}
    {% stylesheet 'banners' %}
  {% endif %}
```
...but that's for another day. 

Making this switch should do two other things:

1. The JavaScript code (that runs based on `GET /api/v1/whoami`) should be looking at the waffle Switch. 
2. We should migrate our existing waffle Flag to waffle Switch. 


Discussion:

Resolving patch:
diff --git a/kuma/core/migrations/0002_auto_20191206_0805.py b/kuma/core/migrations/0002_auto_20191206_0805.py
new file mode 100644
--- /dev/null
+++ b/kuma/core/migrations/0002_auto_20191206_0805.py
@@ -0,0 +1,29 @@
+# -*- coding: utf-8 -*-
+# Generated by Django 1.11.23 on 2019-12-06 08:05
+from __future__ import unicode_literals
+
+from django.db import migrations
+
+
+def move_developer_needs_flag_to_switch(apps, schema_editor):
+    Flag = apps.get_model('waffle', 'Flag')
+    Switch = apps.get_model('waffle', 'Switch')
+    name = 'developer_needs'
+    for flag in Flag.objects.filter(name=name):
+        active = flag.everyone or (flag.percent and flag.percent > 0)
+        Switch.objects.get_or_create(name=name, active=active, note=flag.note)
+        flag.delete()
+
+
+class Migration(migrations.Migration):
+
+    dependencies = [
+        ('core', '0001_squashed_0004_remove_unused_tags'),
+        # This is needed otherwise `apps.get_model('waffle', 'Flag')`
+        # will raise a Django app LookupError.
+        ('waffle', '0001_initial'),
+    ]
+
+    operations = [
+        migrations.RunPython(move_developer_needs_flag_to_switch)
+    ]


Test patch:
Repository: mdn/kuma

Issue:
class="new" is inserted on links that could work
See https://github.com/mdn/kuma/issues/6163#issuecomment-561802945

The document has this raw HTML:
```
<a href="/en-US/docs/Web/Houdini/">Houdini</a>
```
but when you save this, the post-process rendering turns that into:
```
<a class="new" href="/en-US/docs/Web/Houdini/">Houdini</a>
```
But the only reason it claims it doesn't exist is because someone accidentally put in an extra trailing `/`. If you'd just remove that, it should work fine. 


Discussion:

Resolving patch:
diff --git a/kuma/wiki/content.py b/kuma/wiki/content.py
--- a/kuma/wiki/content.py
+++ b/kuma/wiki/content.py
@@ -474,12 +474,17 @@ def __iter__(self):
                                               path_locale=href_locale))
 
                 # Gather up this link for existence check
-                needs_existence_check[locale.lower()][slug.lower()].add(href)
+                slug = slug.lower()
+                if slug.endswith('/'):
+                    # If the slug used in the document has a trailing /
+                    # remove that from here so that it stands a better chance
+                    # to match existing Document slugs.
+                    slug = slug[:-1]
+                needs_existence_check[locale.lower()][slug].add(href)
 
         # Perform existence checks for all the links, using one DB query per
         # locale for all the candidate slugs.
         for locale, slug_hrefs in needs_existence_check.items():
-
             existing_slugs = (Document.objects
                               .filter(locale=locale,
                                       slug__in=slug_hrefs.keys())


Test patch:
diff --git a/kuma/wiki/tests/test_content.py b/kuma/wiki/tests/test_content.py
--- a/kuma/wiki/tests/test_content.py
+++ b/kuma/wiki/tests/test_content.py
@@ -879,6 +879,30 @@ def test_annotate_links_nonexisting_doc(db, anchor, full_url, has_class):
     assert normalize_html(actual_raw) == normalize_html(expected_raw)
 
 
+def test_dont_annotate_links_with_trailing_slashes(db, root_doc):
+    """
+    Links like this: <a href="/en-US/docs/Root/">Root</a>
+    should NOT become this: <a class="new" href="/en-US/docs/Root/">Root</a>
+    """
+    # Sanity check our fixture
+    assert Document.objects.filter(locale='en-US', slug='Root').exists()
+    html = """
+        <ul>
+        <li><a href="/en-US/docs/Root/">Root</a></li>
+        <li><a href="/en-US/docs/Hopeless/">Hopeless</a></li>
+        </ul>
+    """
+    actual_raw = parse(html).annotateLinks(base_url=AL_BASE_URL).serialize()
+    expected_raw = """
+        <ul>
+        <li><a href="/en-US/docs/Root/">Root</a></li>
+        <li><a class="new" rel="nofollow"
+        href="/en-US/docs/Hopeless/">Hopeless</a></li>
+        </ul>
+    """
+    assert normalize_html(actual_raw) == normalize_html(expected_raw)
+
+
 def test_annotate_links_uilocale_to_existing_doc(root_doc):
     """Links to existing docs with embeded locales are unmodified."""
     assert root_doc.get_absolute_url() == '/en-US/docs/Root'
Repository: mdn/kuma

Issue:
Redirect "Featured
Hi there! 

Apologies; it looks like we have another page that needs to be redirected from MDN to Extension Workshop. 

AMO Features Extensions policies (https://developer.mozilla.org/docs/Mozilla/Add-ons/AMO/Policy/Featured) should be redirected to https://extensionworkshop.com/documentation/publish/recommended-extensions/. 

This can happen during the next MDN push. 

Thank you! 




Discussion:
Thanks for ticket @caitmuenster, we'll get this out in our next push!

Resolving patch:
diff --git a/kuma/redirects/redirects.py b/kuma/redirects/redirects.py
--- a/kuma/redirects/redirects.py
+++ b/kuma/redirects/redirects.py
@@ -952,6 +952,8 @@ def locale_redirect(pattern, to, prepend_locale=True, **kwargs):
          'publish/third-party-library-usage/'),
         ('WebExtensions/What_does_review_rejection_mean_to_users',
          'publish/what-does-review-rejection-mean-to-users/'),
+        ('AMO/Policy/Featured',
+         'publish/recommended-extensions/'),
     )
 ]
 


Test patch:
diff --git a/tests/headless/map_301.py b/tests/headless/map_301.py
--- a/tests/headless/map_301.py
+++ b/tests/headless/map_301.py
@@ -563,5 +563,7 @@
          'publish/third-party-library-usage/'),
         ('WebExtensions/What_does_review_rejection_mean_to_users',
          'publish/what-does-review-rejection-mean-to-users/'),
+        ('AMO/Policy/Featured',
+         'publish/recommended-extensions/'),
     )
 ))