internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: mdn/kuma

Issue:
Site search searches on sidebar
Look at this search result: https://developer.mozilla.org/en-US/search?q=django
<img width="726" alt="Screen Shot 2019-12-18 at 8 33 39 AM" src="https://user-images.githubusercontent.com/26739/71090488-2096db80-2171-11ea-999b-a92d470687db.png">

It's showing pages that mention "Django" in an "unexpanded" sidebar. 
That's not helping. It's misleading and if a particular string is in the sidebar, it'll exist on [its own page](https://developer.mozilla.org/en-US/docs/Learn/Server-side/Django). 


Discussion:
The `content` sent to Elasticsearch is here: https://github.com/mdn/kuma/blob/94b36a861dd0bda3610c936a505af9e92b823df5/kuma/wiki/search.py#L79

Resolving patch:
diff --git a/kuma/wiki/search.py b/kuma/wiki/search.py
--- a/kuma/wiki/search.py
+++ b/kuma/wiki/search.py
@@ -76,7 +76,7 @@ def from_django(cls, obj):
             'summary': obj.get_summary_text(),
             'locale': obj.locale,
             'modified': obj.modified,
-            'content': strip_tags(obj.rendered_html or ''),
+            'content': strip_tags(obj.get_body_html() or ''),
             'tags': [o.name for o in obj.tags.all()],
             'kumascript_macros': cls.case_insensitive_keywords(
                 obj.extract.macro_names()),


Test patch:
Repository: mdn/kuma

Issue:
Load modal javascript as defer
Currently, the JavaScript for the auth modal is marked as `async`. As @peterbe mentions here https://github.com/mdn/kuma/pull/6094#discussion_r344937847, that is not ideal. To be honest, I believe mentally my intent was to mark this as `defer` and not `async`.

We should update it to be `defer`


Discussion:
While doing this, we might as well make the following comment in `common.py` more useful:

```
# Multi Auth Switch
```
I would elevate the priority on this because it's disruptive in local development. 
In local development you end up with this:
```html
<script async src="/modal.js"></script>
<script async src="/auth-modal.js"></script>
```
but those two files are not "thread-safe". Periodically (easy to reproduce by refreshing a couple of times) you'll get it that `/auth-modal.js` executes BEFORE `/moda.js` and then you get this:
<img width="524" alt="Screen Shot 2019-12-23 at 3 07 00 PM" src="https://user-images.githubusercontent.com/26739/71381147-53d9d000-25a0-11ea-850a-89914e9e14ed.png">


Resolving patch:
diff --git a/kuma/settings/common.py b/kuma/settings/common.py
--- a/kuma/settings/common.py
+++ b/kuma/settings/common.py
@@ -1026,7 +1026,7 @@ def pipeline_one_scss(slug, **kwargs):
         ),
         'output_filename': 'build/js/auth-modal.js',
         'extra_context': {
-            'async': True,
+            'defer': True,
         },
     },
     'dashboard': {
@@ -1847,7 +1847,9 @@ def get_user_url(user):
 # Whether or not to enable the BCD signalling feature.
 # Affects loading of CSS (statically) and JS (in runtime).
 ENABLE_BCD_SIGNAL = config('ENABLE_BCD_SIGNAL', default=True, cast=bool)
-# Multi Auth Switch
+
+# Enable or disable the multi auth(Google and Github) sign-in flow
+# When disabled, Github will be the default and only Auth provider
 MULTI_AUTH_ENABLED = config('MULTI_AUTH_ENABLED', default=False, cast=bool)
 
 # Content Experiments


Test patch:
Repository: mdn/kuma

Issue:
TypeError in get_seo_description
https://sentry.prod.mozaws.net/operations/mdn-prod/issues/6740633/

```

TypeError: sequence item 0: expected str instance, NoneType found
(21 additional frame(s) were not displayed)
...
  File "kuma/wiki/models.py", line 69, in wrapper
    field_val = fn(self, force_fresh=force_fresh)
  File "kuma/wiki/models.py", line 361, in get_summary_html
    return self.get_summary(strip_markup=False)
  File "kuma/wiki/models.py", line 621, in get_summary
    return get_seo_description(src, self.locale, strip_markup)
  File "newrelic/api/function_trace.py", line 139, in literal_wrapper
    return wrapped(*args, **kwargs)
  File "kuma/wiki/content.py", line 241, in get_seo_description
    seo_summary = ''.join(

TypeError: sequence item 0: expected str instance, NoneType found
```


Discussion:
[Inside `get_seo_description` there's this code](https://github.com/mdn/kuma/blob/ede99b0e2811671c7d6ed44111b452740cbbf7e6/kuma/wiki/content.py#L241-L242):
```python
seo_summary = ''.join(
                    to_html(item) for item in summaryClasses.items())
```
...which implicitly assumes that `to_html(item)` *always* returns a string. 
But it doesn't under certain circumstances. 

The solution would be to comprend why `pg.html(method='html')` sometimes returns `None` instead of a string. 
It can happen if the rendered HTML is this:
```html
<p><span class="seoSummary"></span> Lorem ipsum bla bla.</p>
```


Resolving patch:
diff --git a/kuma/wiki/content.py b/kuma/wiki/content.py
--- a/kuma/wiki/content.py
+++ b/kuma/wiki/content.py
@@ -239,7 +239,7 @@ def get_seo_description(content, locale=None, strip_markup=True):
                 seo_summary = summaryClasses.text()
             else:
                 seo_summary = ''.join(
-                    to_html(item) for item in summaryClasses.items())
+                    to_html(item) or '' for item in summaryClasses.items())
         else:
             paragraphs = page.find('p')
             if paragraphs.length:


Test patch:
diff --git a/kuma/wiki/tests/test_content.py b/kuma/wiki/tests/test_content.py
--- a/kuma/wiki/tests/test_content.py
+++ b/kuma/wiki/tests/test_content.py
@@ -1408,6 +1408,12 @@ def test_multiple_seo_summaries(markup, expected_markup, text, wrapper):
     assert normalize_html(get_seo_description(content, 'en-US', False)) == normalize_html(expected_markup)
 
 
+def test_empty_seo_summary():
+    content = '<p><span class="seoSummary"></span></p>'
+    assert get_seo_description(content, 'en-US', strip_markup=True) == ''
+    assert get_seo_description(content, 'en-US', strip_markup=False) == ''
+
+
 def test_empty_paragraph_content():
     content = """<p></p><div class="overheadIndicator draft draftHeader">
         <strong>DRAFT</strong>
Repository: mdn/kuma

Issue:
Use of access tokens as query parameters in the GitHub OAuth API is deprecated
**Summary**
We use GitHub's OAuth API as one of MDN's sign-up/sign-in OAuth providers. We're starting to receive emails from GitHub that using the https://api.github.com/user API with the access token as a query parameter has been deprecated and that the `Authorization` header should be used instead. This occurs within `GitHubOAuth2Adapter.complete_login` method provided by `django-allauth`, but `django-allauth` has not yet fixed this (although a PR has been submitted that does -- see https://github.com/pennersr/django-allauth/pull/2458). Even if `django-allauth` fixes the issue, it wouldn't help in our case since we override this method (https://github.com/mdn/kuma/blob/266bd9d8ebf24c950037a1965b1967022fca233f/kuma/users/providers/github/views.py#L20). We need to update our overridden method to pass the token via the `Authorization` header rather than via a query parameter.

**Rationale**
We no longer have to concern ourselves with using a deprecated approach.

**Audience**
All users who sign-up/sign-in to MDN via GitHub.



Discussion:

Resolving patch:
diff --git a/kuma/users/providers/github/views.py b/kuma/users/providers/github/views.py
--- a/kuma/users/providers/github/views.py
+++ b/kuma/users/providers/github/views.py
@@ -19,11 +19,11 @@ class KumaGitHubOAuth2Adapter(GitHubOAuth2Adapter):
 
     def complete_login(self, request, app, token, **kwargs):
         session = requests_retry_session()
-        params = {'access_token': token.token}
-        profile_data = session.get(self.profile_url, params=params)
+        headers = {'Authorization': f'token {token.token}'}
+        profile_data = session.get(self.profile_url, headers=headers)
         profile_data.raise_for_status()
         extra_data = profile_data.json()
-        email_data = session.get(self.email_url, params=params)
+        email_data = session.get(self.email_url, headers=headers)
         email_data.raise_for_status()
         extra_data['email_addresses'] = email_data.json()
         return self.get_provider().sociallogin_from_response(request,


Test patch:
Repository: mdn/kuma

Issue:
E - Noindex on /**/docs/MDN/Jobs ?
Similar to https://github.com/mdn/kuma/issues/6422
but this time, instead of `/**/docs/MDN/Doc_status` it's `/**/docs/MDN/Jobs`. 

There are only 9 of them in my month old clone of the production DB.

```
mysql> select locale, slug from wiki_document where slug like 'MDN/Jobs%';
+--------+-------------------------------------------------------+
| locale | slug                                                  |
+--------+-------------------------------------------------------+
| en-US  | MDN/Jobs                                              |
| ja     | MDN/Jobs                                              |
| zh-CN  | MDN/Jobs                                              |
| en-US  | MDN/Jobs/Firefox_developer_content_manager_contractor |
| en-US  | MDN/Jobs/MDN_Developer_Researcher                     |
| ja     | MDN/Jobs/MDN_Developer_Researcher                     |
| zh-CN  | MDN/Jobs/MDN_Developer_Researcher                     |
| en-US  | MDN/Jobs/MDN_technical_writer_contractor              |
| ja     | MDN/Jobs/MDN_technical_writer_contractor              |
+--------+-------------------------------------------------------+
9 rows in set (0.00 sec)
```


So can I add it to [`NOINDEX_SLUG_PREFIXES`](https://github.com/mdn/kuma/blob/3df115a9ba8d3130094a9007f4e9d668a6f274fe/kuma/wiki/constants.py#L668)?



Discussion:
@atopal @chrisdavidmills ?
By the way, I'm going to use the `NOINDEX_SLUG_PREFIXES` in Yari's MySQL importer. 
Basically, MySQL documents that have these slug (prefixes) don't get ported to Git. So far I have the following list:

```javascript
const DEFAULT_EXCLUDE_SLUG_PREFIXES = [
  "User:",
  "Talk:",
  "User_talk:",
  "Template_talk:",
  "Project_talk:",
  "Experiment:",

  // The following come from 'NOINDEX_SLUG_PREFIXES' in
  // https://github.com/mdn/kuma/blob/master/kuma/wiki/constants.py#L668
  // We'll have to manually maintain a match between that and this
  "MDN/Doc_status"
];
``` 
Yeah, you can add them, but I don't quite understand why this is important work right now?
> Yeah, you can add them, but I don't quite understand why this is important work right now?

Because in 2020 we said we'd fix the platform. 

Resolving patch:
diff --git a/kuma/wiki/constants.py b/kuma/wiki/constants.py
--- a/kuma/wiki/constants.py
+++ b/kuma/wiki/constants.py
@@ -665,4 +665,7 @@
 # Any slug that, case *sensitively*, matches this list get the
 # <meta name="robots" content="noindex, nofollow"> HTML header and they get
 # excluded from the sitemap XML files.
-NOINDEX_SLUG_PREFIXES = ("MDN/Doc_status",)
+NOINDEX_SLUG_PREFIXES = (
+    "MDN/Doc_status",
+    "MDN/Jobs",
+)


Test patch:
Repository: mdn/kuma

Issue:
Regression: Math (MathJax/MathML) widgets broken in new platform
The new platform (developer.mozilla.org) cannot correctly present the CKEditor math components that work correctly on wikil.developer.mozilla.org. In particular, I'm using Chrome, and while Chrome doesn't support MathML, the platform is able to render matrices using a simplified form that while it's not ideal is quite readable.

On the new platform, though, the underlying Latex source is output (CKEditor uses Latex input and generates MathML or other syntax as appropriate depending on the browser).

This needs to be fixed because it is rendering pages unreadable, including many key pages in the WebXR and WebGL documentation, plus in a couple of places in the WebRTC docs.


Discussion:
I believe I hunted this one down once before and there is some CSS and or JS that is missing. @a2sheppy can you please add an example page here? Thank you.
Sorry about forgetting to include examples. Here we go:

* https://developer.mozilla.org/en-US/docs/Web/API/WebXR_Device_API/Cameras#Positioning_the_camera
* https://developer.mozilla.org/en-US/docs/Web/API/XRRigidTransform/matrix
* https://developer.mozilla.org/en-US/docs/Web/API/CanvasRenderingContext2D/getTransform
* https://developer.mozilla.org/en-US/docs/Web/API/RTCIceCandidateStats/priority
* https://developer.mozilla.org/en-US/docs/Web/Media/Formats/Audio_concepts
* https://developer.mozilla.org/en-US/docs/Web/CSS/transform-function/translateZ (in the table under "Values")
put this as a p2 and possibly something for this sprint @atopal?
yes

Resolving patch:
diff --git a/kuma/settings/common.py b/kuma/settings/common.py
--- a/kuma/settings/common.py
+++ b/kuma/settings/common.py
@@ -903,6 +903,11 @@ def pipeline_one_scss(slug, **kwargs):
         "output_filename": "build/js/banners.js",
         "extra_context": {"async": True},
     },
+    "mathml": {
+        "source_filenames": ("js/components/mathml.js",),
+        "output_filename": "build/js/mathml.js",
+        "extra_context": {"defer": True},
+    },
     "users": {
         "source_filenames": ("js/users.js",),
         "output_filename": "build/js/users.js",


Test patch:
Repository: mdn/kuma

Issue:
[Django 2]: Address collision with custom view_document permission
> Blocks: #6172 (Upgrade to Django 2)

Django 2.1 introduces a new `view` permission on models, which conflicts with our legacy `view_document` permission. This is reported by `manage.py check` in Django 2 as:

> wiki.Document: (auth.E005) The permission codenamed 'view_document' clashes with a builtin permission for model 'wiki.Document'.

This causes the majority of our test suite to fail:

    468 passed, 1503 errors

We need to remove or rename our `view_document` permission to resolve this error.

See: https://docs.djangoproject.com/en/2.2/releases/2.1/#model-view-permission


Discussion:

Resolving patch:
diff --git a/kuma/wiki/migrations/0007_view_document_permission.py b/kuma/wiki/migrations/0007_view_document_permission.py
new file mode 100644
--- /dev/null
+++ b/kuma/wiki/migrations/0007_view_document_permission.py
@@ -0,0 +1,44 @@
+from django.core.exceptions import ObjectDoesNotExist
+from django.db import migrations
+
+
+def remove_view_document_permission(apps, schema_editor):
+    """
+    Delete our legacy view_document permission, which conflicts with the default
+    permissions added by Django 2.1.
+    """
+    Permission = apps.get_model("auth.Permission")
+    ContentType = apps.get_model("contenttypes.ContentType")
+
+    try:
+        content_type = ContentType.objects.get(app_label="wiki", model="document")
+        Permission.objects.get(
+            content_type=content_type, codename=("view_document")
+        ).delete()
+    except ObjectDoesNotExist:
+        # The ContentType and Permission may not exist on a new, empty database,
+        # like in our CI environment. And that's OK.
+        pass
+
+
+class Migration(migrations.Migration):
+
+    dependencies = [
+        ("wiki", "0006_auto_20191023_0741"),
+    ]
+
+    operations = [
+        migrations.AlterModelOptions(
+            name="document",
+            options={
+                "permissions": (
+                    ("move_tree", "Can move a tree of documents"),
+                    ("purge_document", "Can permanently delete document"),
+                    ("restore_document", "Can restore deleted document"),
+                )
+            },
+        ),
+        migrations.RunPython(
+            remove_view_document_permission, migrations.RunPython.noop
+        ),
+    ]
diff --git a/kuma/wiki/models.py b/kuma/wiki/models.py
--- a/kuma/wiki/models.py
+++ b/kuma/wiki/models.py
@@ -324,7 +324,6 @@ class Meta(object):
             ("slug", "locale"),
         )
         permissions = (
-            ("view_document", "Can view document"),
             ("move_tree", "Can move a tree of documents"),
             ("purge_document", "Can permanently delete document"),
             ("restore_document", "Can restore deleted document"),


Test patch:
Repository: mdn/kuma

Issue:
Possibly to prefill Reason textarea on $delete
If you're going to have some human-helped automation that deletes the 20% or so non-en-US documents that aren't actually translated, it would be nice if you don't have to type in the same reason every time. 
<img width="989" alt="Screen Shot 2020-02-26 at 11 56 40 AM" src="https://user-images.githubusercontent.com/26739/75367987-1be85500-588f-11ea-8ba1-f49e0db69cc7.png">

Would be neat if you could control it with something like `?reason=Sample%20reason`


Discussion:

Resolving patch:
diff --git a/kuma/wiki/views/delete.py b/kuma/wiki/views/delete.py
--- a/kuma/wiki/views/delete.py
+++ b/kuma/wiki/views/delete.py
@@ -100,7 +100,8 @@ def delete_document(request, document_slug, document_locale):
             document.delete()
             return redirect(document)
     else:
-        form = DocumentDeletionForm()
+
+        form = DocumentDeletionForm(initial={"reason": request.GET.get("reason", "")})
 
     context = {
         "document": document,


Test patch:
Repository: mdn/kuma

Issue:
AttributeError inside build_json_data()
**Summary**
https://sentry.prod.mozaws.net/operations/mdn-prod/issues/6218678/?query=is:unresolved

What happens is that some documents don't have a `current_revision`. 
[This code](https://github.com/mdn/kuma/blob/34aeef7f85b294c7392d791b4084a28354f8efc2/kuma/wiki/models.py#L654-L658) assumes and relies on every document having a `current_revision`. If because if it's `None` you can't do `revision.summary` for example. 

In production there are only 3 that are in this weird state:
```
>>> Document.all_objects.filter(current_revision__isnull=True).count()
3
>>> Document.all_objects.filter(current_revision__isnull=True)
<QuerySet [<Document: /ko/docs/Web/API/RTCPeerConnection/createOffer (RTCPeerConnection.createOffer())>, <Document: /pt-BR/docs/Glossario/Head (Head)>, <Document: /ko/docs/Learn/CSS/CSS_layout/%EB%A0%88%EA%B1%B0%EC%8B%9C_%EC%A1%B0%ED%8C%90_%EB%A9%94%EC%84%9C%EB%93%9C (레거시_조판_메서드)>]>
```

I have several more in my local database which I *think* is due to some weird bug (maybe) in the `scrape_document`. 

I actually don't know how to trigger the problem. I tried to Shift-Refresh on some prod wiki pages that are the parents of these weird documents. 

WHen I was working on this, it prevented the KumaScript rendering to work so I put in this diff:

```diff
diff --git a/kuma/wiki/models.py b/kuma/wiki/models.py
index 4153b170a..e6e7ab616 100644
--- a/kuma/wiki/models.py
+++ b/kuma/wiki/models.py
@@ -652,6 +652,8 @@ class Document(NotificationsMixin, models.Model):
         if self.pk:
             for translation in self.other_translations:
                 revision = translation.current_revision
+                if not revision:
+                    continue
                 if revision.summary:
                     summary = revision.summary
                 else:

```

That makes it NOT fail anymore. 


Discussion:
Sentry issue: [MDN-PROD-WR](https://sentry.prod.mozaws.net/operations/mdn-prod/issues/6218678/)

Resolving patch:
diff --git a/kuma/wiki/models.py b/kuma/wiki/models.py
--- a/kuma/wiki/models.py
+++ b/kuma/wiki/models.py
@@ -652,6 +652,8 @@ def build_json_data(self):
         if self.pk:
             for translation in self.other_translations:
                 revision = translation.current_revision
+                if not revision:
+                    continue
                 if revision.summary:
                     summary = revision.summary
                 else:


Test patch:
Repository: mdn/kuma

Issue:
T - Add email to whoami userData so that Stripe form can be prefilled. 



Discussion:

Resolving patch:
diff --git a/kuma/api/v1/views.py b/kuma/api/v1/views.py
--- a/kuma/api/v1/views.py
+++ b/kuma/api/v1/views.py
@@ -222,6 +222,7 @@ def whoami(request):
             "is_subscriber": UserSubscription.objects.filter(
                 user=user, canceled__isnull=True
             ).exists(),
+            "email": user.email,
         }
     else:
         data = {
@@ -233,6 +234,7 @@ def whoami(request):
             "is_beta_tester": False,
             "avatar_url": None,
             "is_subscriber": False,
+            "email": None,
         }
 
     # Add waffle data to the dict we're going to be returning.


Test patch:
diff --git a/kuma/api/v1/tests/test_views.py b/kuma/api/v1/tests/test_views.py
--- a/kuma/api/v1/tests/test_views.py
+++ b/kuma/api/v1/tests/test_views.py
@@ -223,6 +223,7 @@ def test_whoami_anonymous(client, settings, timezone):
             "samples": {"sample_always": True, "sample_never": False},
         },
         "is_subscriber": False,
+        "email": None,
     }
     assert_no_cache_header(response)
 
@@ -284,6 +285,7 @@ def test_whoami(
             "samples": {"sample_always": True, "sample_never": False},
         },
         "is_subscriber": False,
+        "email": "wiki_user@example.com",
     }
     assert_no_cache_header(response)
Repository: mdn/kuma

Issue:
SystemError: <method 'get' of 'dict' objects> returned a result with an error set
https://sentry.prod.mozaws.net/operations/mdn-prod/issues/6659909/

```
timeout: timeout
SystemError: <method 'get' of 'dict' objects> returned a result with an error set
  File "meinheld/mlogging.py", line 187, in _access
    'h': environ.get('REMOTE_ADDR', '-'),

SystemError: <method 'get' of 'dict' objects> returned a result with an error set
```

Low priority: 12x in 2yrs, but might worth looking into with spare time.


Discussion:
Steps to *maybe* reproduce: Open http://wiki.localhost.org:8000/en-US/docs/get-documents?term=a
The SQL query is actually not too bad but the response is just WAY to large. That URL above creates a JSON response, on my local docker-compose, that is 4.3MB large. 
It's actually somewhat important because triggering that will consume such a large payload that it takes up that whole Python process for a long time. 

A simple solution would be to add some basic pagination. I.e. only the first 100 documents. 
Actually, `/$locale/docs/get-documents` is not used anywhere. 
I think we should just delete it. Or at least break it and see who screams. 
~~@a2sheppy Do you know if this is used anywhere?~~

Ignore me. 
Actually! I was wrong. It is used in wiki-move.js and wiki-edit.js
So I think the next right thing to do is to use pagination. E.g. only return top 100 matches. 

Resolving patch:
diff --git a/kuma/wiki/views/misc.py b/kuma/wiki/views/misc.py
--- a/kuma/wiki/views/misc.py
+++ b/kuma/wiki/views/misc.py
@@ -84,7 +84,7 @@ def autosuggest_documents(request):
 
     # Generates a list of acceptable docs
     docs_list = []
-    for doc in docs:
+    for doc in docs[:100]:
         data = doc.get_json_data()
         data["label"] += " [" + doc.locale + "]"
         docs_list.append(data)


Test patch: