internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: mdn/kuma

Issue:
Add support for Cross-Origin search requests
## Description

I would like to be able to query the MDN Docs as documented [here](https://developer.mozilla.org/en-US/docs/MDN/Tools/Search#Search_query_format)

## Reason

I am currently developing a front-end component which would be able to search the MDN docs, and as it's hosted on a different domain from the MDN docs, a CORS error ensues

## Steps to re-produce:

- Open Dev Console on a page that isn't under `developer.mozilla.org`
- Run `fetch("https://developer.mozilla.org/api/v1/search/en-US?q=History%20Api")`

#### What should happen

A successful request should go through and return JSON data

#### What happens

An error in the request with the classic CORS error message

```
Access to fetch at 'https://developer.mozilla.org/api/v1/search/en-US?q=History%20Api' from origin 'https://matsu.fi' has been blocked by CORS policy: No 'Access-Control-Allow-Origin' header is present on the requested resource. If an opaque response serves your needs, set the request's mode to 'no-cors' to fetch the resource with CORS disabled.
```


Discussion:
I see no reason *not* to do this. Any objections @escattone ?

@peterbe I agree. I think we should do this.

Resolving patch:
diff --git a/kuma/api/v1/decorators.py b/kuma/api/v1/decorators.py
new file mode 100644
--- /dev/null
+++ b/kuma/api/v1/decorators.py
@@ -0,0 +1,14 @@
+from functools import wraps
+
+
+def allow_CORS_GET(func):
+    """Decorator to allow CORS for GET requests"""
+
+    @wraps(func)
+    def inner(request, *args, **kwargs):
+        response = func(request, *args, **kwargs)
+        if "GET" == request.method:
+            response["Access-Control-Allow-Origin"] = "*"
+        return response
+
+    return inner
diff --git a/kuma/api/v1/views.py b/kuma/api/v1/views.py
--- a/kuma/api/v1/views.py
+++ b/kuma/api/v1/views.py
@@ -27,6 +27,7 @@
 from waffle.decorators import waffle_flag
 from waffle.models import Flag, Sample, Switch
 
+from kuma.api.v1.decorators import allow_CORS_GET
 from kuma.api.v1.serializers import BCSignalSerializer, UserDetailsSerializer
 from kuma.core.email_utils import render_email
 from kuma.core.ga_tracking import (
@@ -149,7 +150,7 @@ class APISearchView(SearchView):
     )
 
 
-search = never_cache(APISearchView.as_view())
+search = never_cache(allow_CORS_GET(APISearchView.as_view()))
 
 
 @ratelimit(key="user_or_ip", rate="10/d", block=True)


Test patch:
diff --git a/kuma/api/v1/tests/test_views.py b/kuma/api/v1/tests/test_views.py
--- a/kuma/api/v1/tests/test_views.py
+++ b/kuma/api/v1/tests/test_views.py
@@ -177,6 +177,7 @@ def test_search_basic(self):
         response = self.client.get(url, {"q": "article"})
         assert response.status_code == 200
         assert response["content-type"] == "application/json"
+        assert response["Access-Control-Allow-Origin"] == "*"
         data = response.json()
         assert data["documents"]
         assert data["count"] == 4
Repository: mdn/kuma

Issue:
Make /$locale/search.json redirect
**Summary**
We get a lot of `NoReverseMatch` on URLs like https://developer.mozilla.org/en-US/search.json
That endpoint disappeared when we switched to the new search API. 
Let's make it redirect. 


**Additional context**
https://sentry.prod.mozaws.net/operations/mdn-prod/issues/10482841/



Discussion:
People WILL be miffed/surprised/angry/confused about the fact that the JSON structure becomes different when we make this redirect. 

Resolving patch:
diff --git a/kuma/search/views.py b/kuma/search/views.py
--- a/kuma/search/views.py
+++ b/kuma/search/views.py
@@ -1,5 +1,6 @@
 from urllib.parse import parse_qs, urlencode
 
+from django.conf import settings
 from django.shortcuts import render
 from django.urls import reverse_lazy
 from django.views.decorators.cache import never_cache
@@ -115,11 +116,14 @@ class SearchRedirectView(RedirectView):
 
     def get_redirect_url(self, *args, **kwargs):
         query_string = self.request.META.get("QUERY_STRING")
-        url = reverse_lazy(
-            "api.v1.search", kwargs={"locale": self.request.LANGUAGE_CODE}
-        )
-        if query_string:
-            url += "?" + query_string
+        url = reverse_lazy("api.v1.search")
+        qs = parse_qs(query_string)
+        # If you used `/en-Us/search.json` you can skip the `?locale=`
+        # because the default locale in `/api/v1/search` is `en-US`.
+        if self.request.LANGUAGE_CODE.lower() != settings.LANGUAGE_CODE.lower():
+            qs["locale"] = self.request.LANGUAGE_CODE
+        if qs:
+            url += "?" + urlencode(qs, True)
         return url
 
 


Test patch:
Repository: mdn/kuma

Issue:
Locale searches aren't working
**Summary**
https://developer.mozilla.org/de/search?q=Zuweisungsoperator yields nothing. 
But it should work, because of: https://developer.mozilla.org/api/v1/search?q=Zuweisungsoperator&locale=de


**Additional context**
_Is there anything else we should know?_



Discussion:

Resolving patch:
diff --git a/kuma/search/views.py b/kuma/search/views.py
--- a/kuma/search/views.py
+++ b/kuma/search/views.py
@@ -46,7 +46,7 @@ def search(request, *args, **kwargs):
         # it to do a search in both locales.
         # This might come in handy for people searching in a locale where
         # there's very little results but they'd be happy to get the en-US ones.
-        response = search_api(request, *args, **kwargs)
+        response = search_api(request, locale=request.LANGUAGE_CODE, *args, **kwargs)
         results = response.data
 
         error = None


Test patch:
Repository: mdn/kuma

Issue:
Search results presents the locale in the wrong case
**Summary**
E.g. https://developer.mozilla.org/en-US/search?q=mdn+contribute+
See screenshot:
<img width="932" alt="Screen Shot 2021-02-04 at 10 59 44 AM" src="https://user-images.githubusercontent.com/26739/106919753-6cd80e80-66d8-11eb-97a1-d409dfc2e36b.png">


**Additional context**

The Yari site-search is coming. But this might be easy to fix. 



Discussion:

Resolving patch:
diff --git a/kuma/search/views.py b/kuma/search/views.py
--- a/kuma/search/views.py
+++ b/kuma/search/views.py
@@ -89,6 +89,21 @@ def search(request, *args, **kwargs):
                     query_string_parsed["page"] = f"{page - 1}"
                 previous_url = f"?{urlencode(query_string_parsed, True)}"
 
+            def package_document(document):
+                # The `results['documents']` will have the `locale` in lowercase.
+                # That's good for searching but now what we want to display.
+                # Here in Kuma we can't use the `mdn_url` so to get that right
+                # we have to manually correct that.
+                locale, slug = document["mdn_url"][1:].split("/docs/")
+                data = {
+                    "title": document["title"],
+                    "slug": slug,
+                    "locale": locale,
+                    "summary": document["summary"],
+                    "excerpt": "<br>".join(document["highlight"].get("body", [])),
+                }
+                return data
+
             results = {
                 "count": count,
                 "next": next_url,
@@ -96,16 +111,7 @@ def search(request, *args, **kwargs):
                 "query": request.GET.get("q"),
                 "start": (page - 1) * size + 1,
                 "end": page * size,
-                "documents": [
-                    {
-                        "title": x["title"],
-                        "slug": x["slug"],
-                        "locale": x["locale"],
-                        "summary": x["summary"],
-                        "excerpt": "<br>".join(x["highlight"].get("body", [])),
-                    }
-                    for x in results["documents"]
-                ],
+                "documents": [package_document(x) for x in results["documents"]],
             }
 
         context = {"results": {"results": None if error else results, "error": error}}


Test patch:
Repository: mdn/kuma

Issue:
The "Did you mean..." suggestions' total is wrong
The "Did you mean..." says there are supposed to be 103 matches on this suggestion:
<img width="503" alt="Screen Shot 2021-02-09 at 5 21 50 PM" src="https://user-images.githubusercontent.com/26739/107436375-6fcd6780-6afb-11eb-8989-a6ea605881eb.png">

But if you apply it, you get a different total:
<img width="458" alt="Screen Shot 2021-02-09 at 5 22 00 PM" src="https://user-images.githubusercontent.com/26739/107436459-8e336300-6afb-11eb-90bf-7e19fdddf08d.png">

Not sure if it's an error in Kuma or in how the results are presented in Yari. 



Discussion:
It's a problem with the Kuma code. Yari displays the numbers correctly. It's just the wrong number.
<img width="657" alt="Screen Shot 2021-02-09 at 5 45 33 PM" src="https://user-images.githubusercontent.com/26739/107438538-9e990d00-6afe-11eb-9131-e3ff4eb4390a.png">

Transferring this to Kuma. 

Resolving patch:
diff --git a/kuma/api/v1/search/__init__.py b/kuma/api/v1/search/__init__.py
--- a/kuma/api/v1/search/__init__.py
+++ b/kuma/api/v1/search/__init__.py
@@ -215,7 +215,7 @@ def _find(params, total_only=False, make_suggestions=False, min_suggestion_score
             if score > min_suggestion_score or 1:
                 # Sure, this is different way to spell, but what will it yield
                 # if you actually search it?
-                total = _find(params, total_only=True)
+                total = _find(dict(params, query=string), total_only=True)
                 if total["value"] > 0:
                     suggestions.append(
                         {


Test patch:
Repository: mdn/kuma

Issue:
Sign-up doesn't work for private GitHub emails
We can use the mdnwebdocs.tester GitHub account to try to replicate the bug. 



Discussion:

Resolving patch:
diff --git a/kuma/users/views.py b/kuma/users/views.py
--- a/kuma/users/views.py
+++ b/kuma/users/views.py
@@ -685,9 +685,22 @@ def get_form(self, form_class=None):
 
         # This magic sauce only exists whilst this Django view is being used
         # both by Yari (in testing) and by old Kuma (production).
-        # Once Kuma id divorced from doing any
+        # Once Kuma is divorced from doing any HTML rendering, we can refactor this.
         is_yari_signup = self.request.session.get("yari_signup", False)
         if is_yari_signup:
+
+            if not email and extra_email_addresses:
+                # Pick the first primary email.
+                for data in extra_email_addresses:
+                    if data["primary"]:
+                        email = data["email"]
+                        break
+                else:
+                    # Pick the first non-primary email.
+                    email = extra_email_addresses[0]["email"]
+
+            form.initial["email"] = email
+
             form.data = form.data.copy()
             for key in form.initial:
                 if key not in form.data and form.initial[key]:
@@ -703,7 +716,6 @@ def form_valid(self, form):
         We send our welcome email via celery during complete_signup.
         So, we need to manually commit the user to the db for it.
         """
-
         selected_email = form.cleaned_data["email"]
         if selected_email in self.email_addresses:
             data = self.email_addresses[selected_email]


Test patch:
Repository: mdn/kuma

Issue:
Add country geo information to /api/v1/whoami
We can use this immediately to be able to determine if a user should be seeing certain banners in Yari. 


Discussion:

Resolving patch:
diff --git a/kuma/api/v1/views.py b/kuma/api/v1/views.py
--- a/kuma/api/v1/views.py
+++ b/kuma/api/v1/views.py
@@ -82,6 +82,15 @@ def whoami(request):
     else:
         data = {}
 
+    geo = {}
+    # https://aws.amazon.com/about-aws/whats-new/2020/07/cloudfront-geolocation-headers/
+    cloudfront_country_header = "HTTP_CLOUDFRONT_VIEWER_COUNTRY_NAME"
+    cloudfront_country_value = request.META.get(cloudfront_country_header)
+    if cloudfront_country_value:
+        geo["country"] = cloudfront_country_value
+    if geo:
+        data["geo"] = geo
+
     data["waffle"] = {
         "flags": {},
         "switches": {s.name: True for s in Switch.get_all() if s.is_active()},


Test patch:
diff --git a/kuma/api/v1/tests/test_forms.py b/kuma/api/v1/tests/test_forms.py
--- a/kuma/api/v1/tests/test_forms.py
+++ b/kuma/api/v1/tests/test_forms.py
@@ -27,7 +27,6 @@ def test_search_form_locale_happy_path():
     request = RequestFactory().get("/api/v1/search?q=foo&locale=Fr&locale=de")
     form = SearchForm(request.GET, initial=initial)
     assert form.is_valid()
-    print(form.cleaned_data)
     assert form.cleaned_data["locale"] == ["Fr", "de"]
 
     # Note, same as the initial default
diff --git a/kuma/api/v1/tests/test_views.py b/kuma/api/v1/tests/test_views.py
--- a/kuma/api/v1/tests/test_views.py
+++ b/kuma/api/v1/tests/test_views.py
@@ -59,6 +59,16 @@ def test_whoami_anonymous(client, settings):
     assert_no_cache_header(response)
 
 
+@pytest.mark.django_db
+def test_whoami_anonymous_cloudfront_geo(client, settings):
+    """Test response for anonymous users."""
+    url = reverse("api.v1.whoami")
+    response = client.get(url, HTTP_CLOUDFRONT_VIEWER_COUNTRY_NAME="US of A")
+    assert response.status_code == 200
+    assert response["content-type"] == "application/json"
+    assert response.json()["geo"] == {"country": "US of A"}
+
+
 @pytest.mark.django_db
 @pytest.mark.parametrize(
     "is_staff,is_superuser,is_beta_tester",
Repository: mdn/kuma

Issue:
Refactorings for welcome HTML email 
Based on https://github.com/mdn/kuma/pull/7866 
we'll drop the `utm` query string things and we'll get rid of the plain text email template. 


Discussion:

Resolving patch:
diff --git a/kuma/users/tasks.py b/kuma/users/tasks.py
--- a/kuma/users/tasks.py
+++ b/kuma/users/tasks.py
@@ -5,6 +5,7 @@
 from django.contrib.auth import get_user_model
 from django.utils import translation
 from django.utils.translation import gettext_lazy as _
+from pyquery import PyQuery as pq
 
 from kuma.core.decorators import skip_in_maintenance_mode
 from kuma.core.email_utils import render_email
@@ -48,8 +49,18 @@ def send_welcome_email(user_pk, locale):
         context = {"username": user.username}
         log.debug("Using the locale %s to send the welcome email", locale)
         with translation.override(locale):
-            content_plain = render_email("users/email/welcome/plain.ltxt", context)
             content_html = render_email("users/email/welcome/html.ltxt", context)
+            doc = pq(content_html)
+            urls = []
+            for i, link in enumerate(doc("body a[href]").items()):
+                link.text(f"{link.text()}[{i + 1}]")
+                urls.append((i + 1, link.attr("href")))
+
+            content_plain = doc("body").text().replace("\n", "\n\n")
+            if urls:
+                content_plain += "\n\n"
+                for i, url in urls:
+                    content_plain += f"[{i}] {url}\n"
 
             email = EmailMultiAlternativesRetrying(
                 _("Getting started with your new MDN account"),


Test patch:
diff --git a/kuma/users/tests/test_tasks.py b/kuma/users/tests/test_tasks.py
--- a/kuma/users/tests/test_tasks.py
+++ b/kuma/users/tests/test_tasks.py
@@ -48,7 +48,6 @@ def test_default_language_email(self):
         welcome_email = mail.outbox[0]
         expected_to = [testuser.email]
         self.assertEqual(expected_to, welcome_email.to)
-        self.assertTrue("utm_campaign=welcome" in welcome_email.body)
 
     @mock.patch("kuma.users.tasks.strings_are_translated")
     def test_dont_send_untranslated_language_email(self, strings_are_translated):
@@ -88,7 +87,6 @@ def test_welcome_mail_for_verified_email(self):
         welcome_email = mail.outbox[0]
         expected_to = [testuser.email]
         self.assertEqual(expected_to, welcome_email.to)
-        self.assertTrue("utm_campaign=welcome" in welcome_email.body)
 
     @override_switch("welcome_email", True)
     @call_on_commit_immediately
@@ -131,7 +129,6 @@ def test_welcome_mail_for_unverified_email(self):
         welcome_email = mail.outbox[1]
         expected_to = [email_address.email]
         self.assertEqual(expected_to, welcome_email.to)
-        self.assertTrue("utm_campaign=welcome" in welcome_email.body)
 
         # now add second unverified email address to the user
         # and check if the usual confirmation email is sent out
Repository: mdn/kuma

Issue:
bug 1314290: Update to Django 1.8.16
Django 1.8.15 -> 1.8.16 - Security release

https://www.djangoproject.com/weblog/2016/nov/01/security-releases/


Discussion:
## [Current coverage](https://codecov.io/gh/mozilla/kuma/pull/4024?src=pr) is 86.10% (diff: 100%)

> Merging [#4024](https://codecov.io/gh/mozilla/kuma/pull/4024?src=pr) into [master](https://codecov.io/gh/mozilla/kuma/branch/master?src=pr) will not change coverage

``` diff
@@             master      #4024   diff @@
==========================================
  Files           146        146          
  Lines          8926       8926          
  Methods           0          0          
  Messages          0          0          
  Branches       1193       1193          
==========================================
  Hits           7686       7686          
  Misses         1002       1002          
  Partials        238        238          
```

> Powered by [Codecov](https://codecov.io?src=pr). Last update [5db5f23...0b34fe5](https://codecov.io/gh/mozilla/kuma/compare/5db5f235006c4517c807964f8f6b8bd8044f5f79...0b34fe5f4e4f21c3f48134f9a7b92148499b7a13?src=pr)

the security update seems like not affecting any of the components of kuma.
As the tests pass, r+ 


Resolving patch:
diff --git a/kuma/api/v1/search/__init__.py b/kuma/api/v1/search/__init__.py
--- a/kuma/api/v1/search/__init__.py
+++ b/kuma/api/v1/search/__init__.py
@@ -110,15 +110,29 @@ def _find(params, total_only=False, make_suggestions=False, min_suggestion_score
             "body_suggestions", params["query"], term={"field": "body"}
         )
 
+    # The business logic here that we search for things different ways,
+    # and each different way as a different boost which dictates its importance.
+    # The importance order is as follows:
+    #
+    #  1. Title match-phrase
+    #  2. Title match
+    #  3. Body match-phrase
+    #  4. Body match
+    #
+    # The order is determined by the `boost` number in the code below.
+    # Remember that sort order is a combination of "match" and popularity, but
+    # ideally the popularity should complement. Try to get a pretty good
+    # sort by pure relevance first, and let popularity just make it better.
+    #
     sub_queries = []
-    sub_queries.append(Q("match", title={"query": params["query"], "boost": 2.0}))
+    sub_queries.append(Q("match", title={"query": params["query"], "boost": 5.0}))
     sub_queries.append(Q("match", body={"query": params["query"], "boost": 1.0}))
     if " " in params["query"]:
         sub_queries.append(
             Q("match_phrase", title={"query": params["query"], "boost": 10.0})
         )
         sub_queries.append(
-            Q("match_phrase", body={"query": params["query"], "boost": 5.0})
+            Q("match_phrase", body={"query": params["query"], "boost": 2.0})
         )
 
     sub_query = query.Bool(should=sub_queries)


Test patch:
Repository: web2py/web2py

Issue:
Rocket has problems with large uploads from wininet client
If files are greater than about 1M, Rocket has delays posting the upload, and may show a timeout in the log.  User thinks transfer took a few seconds and was successful.

Reported in
<URL:https://groups.google.com/d/msg/web2py/dTK5lxWBsIA/JwskeLCrBwAJ>
and earlier in
<URL:https://groups.google.com/d/msg/web2py/lSdsqHLftVQ/Hf1j7z5jCwAJ>

web2py 2.14.6 on AWS Linux

/dps



Discussion:

Resolving patch:
diff --git a/gluon/dal.py b/gluon/dal.py
--- a/gluon/dal.py
+++ b/gluon/dal.py
@@ -1767,12 +1767,17 @@ def geoexpand(field):
         sql_w = ' WHERE ' + self.expand(query) if query else ''
 
         if inner_join and not left:
-            sql_t = ', '.join([self.table_alias(t)
+            # Wrap table references with parenthesis (approach 1)
+            # sql_t = ', '.join([self.table_alias(t)
+            #                    for t in iexcluded + itables_to_merge.keys()])
+            # sql_t = '(%s)' % sql_t
+            # or approach 2: Use 'JOIN' instead comma:
+            sql_t = ' JOIN '.join([self.table_alias(t)
                                for t in iexcluded + itables_to_merge.keys()])
             for t in ijoinon:
                 sql_t += ' %s %s' % (icommand, t)
         elif not inner_join and left:
-            sql_t = ', '.join([self.table_alias(t)
+            sql_t = ' JOIN '.join([self.table_alias(t)
                                for t in excluded + tables_to_merge.keys()])
             if joint:
                 sql_t += ' %s %s' % (command,
@@ -1785,7 +1790,7 @@ def geoexpand(field):
             tables_in_joinon = set(joinont + ijoinont)
             tables_not_in_joinon = \
                 all_tables_in_query.difference(tables_in_joinon)
-            sql_t = ','.join([self.table_alias(t) for t in tables_not_in_joinon])
+            sql_t = ' JOIN '.join([self.table_alias(t) for t in tables_not_in_joinon])
             for t in ijoinon:
                 sql_t += ' %s %s' % (icommand, t)
             if joint:


Test patch:
Repository: web2py/web2py

Issue:
db.export_to_csv_file() error with python 3.7
**Describe the bug**
db.export_to_csv_file() error with python 3.7
```
TypeError: a bytes-like object is required, not 'str'
```
in
```
web2py/gluon/packages/dal/pydal/base.py", line 844, in export_to_csv_file
    ofile.write('TABLE %s\r\n' % table)
```

**To Reproduce**
```
Bens-iMac:web2py ben$ pipenv run python web2py.py -M -S pacificW
/Users/ben/.local/share/virtualenvs/web2py-6f1t1lX6/lib/python3.7/site-packages/psycopg2/__init__.py:144: UserWarning: The psycopg2 wheel package will be renamed from release 2.8; in order to keep installing from binary please use "pip install psycopg2-binary" instead. For details see: <http://initd.org/psycopg/docs/install.html#binary-install-from-pypi>.
  """)
web2py Web Framework
Created by Massimo Di Pierro, Copyright 2007-2018
Version 2.17.1-stable+timestamp.2018.08.05.17.57.00
Database drivers available: sqlite3, psycopg2, imaplib, pymysql, pyodbc
WARNING:web2py:import IPython error; use default python shell
Python 3.7.0 (default, Jul 23 2018, 20:22:55) 
[Clang 9.1.0 (clang-902.0.39.2)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
(InteractiveConsole)
>>> db.export_to_csv_file(open('somefile.csv', 'wb'))
Traceback (most recent call last):
  File "<console>", line 1, in <module>
  File "/Users/ben/Documents/programs/web2py/gluon/packages/dal/pydal/base.py", line 844, in export_to_csv_file
    ofile.write('TABLE %s\r\n' % table)
TypeError: a bytes-like object is required, not 'str'
>>> 
```


Discussion:

Resolving patch:
diff --git a/gluon/dal/adapters/oracle.py b/gluon/dal/adapters/oracle.py
--- a/gluon/dal/adapters/oracle.py
+++ b/gluon/dal/adapters/oracle.py
@@ -136,7 +136,7 @@ def execute(self, command, args=None):
         return self.log_execute(command, args)
 
     def create_sequence_and_triggers(self, query, table, **args):
-        tablename = table._tablename
+        tablename = table._rname or table._tablename
         id_name = table._id.name
         sequence_name = table._sequence_name
         trigger_name = table._trigger_name


Test patch: