internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: pypi/warehouse

Issue:
Sorting searches by 'Date last updated' results in 503
**Describe the bug**

When trying to search for anything on pypi.org, sorting by relevance or trending works fine, but sorting by date last updated returns a 503 error.

**Expected behavior**

Search results, sorted by date.

**To Reproduce**

Example URL: https://pypi.org/search/?q=test&o=-created

Result:

> Sorry, something went wrong
> 
> PyPI is down for maintenance or is having an outage.
> 
> This is affecting several of our services, including our web interface.
> If you are trying to install a package, you should be able to pip install packages without problem.
> 
>     Check our status page, or
>     View Python Status on Twitter

The status page, though, shows all green.


**My Platform**

- Win 10, Firefox 66.0.3
- Ubuntu 18.04, Chrome 74.0.3729.108

---

Edit: I know this functionality was working at least as recently as last Thursday, 2 May 2019.


Discussion:
This is the problem with `formatting date` to human understandable format, I'm fixing this. will submit a PR soon.
This started occurring May 5, 2019 5:38:49 PM UTC, did we merge something that's causing this?
I think I merged an elasticsearch update yea.

Resolving patch:
diff --git a/warehouse/packaging/search.py b/warehouse/packaging/search.py
--- a/warehouse/packaging/search.py
+++ b/warehouse/packaging/search.py
@@ -75,3 +75,8 @@ def from_db(cls, release):
         obj["zscore"] = release.zscore
 
         return obj
+
+    class Index:
+        # make sure this class can match any index so it will always be used to
+        # deserialize data coming from elasticsearch.
+        name = "*"


Test patch:
Repository: pypi/warehouse

Issue:
Expose TOTP provisioning code for manual entry
I use the command line tool "oathtool" for my 2FA.  But your 2FA enrollment page only shows a QR code.  How do I get the key for oathtool use?


Discussion:
Hi @garyemiller thanks for opening this ticket.

Currently we don't expose this information in the user interface, but we can take a look into this.  

@woodruffw I assume we can simply output this information on the provisioning page?  If you want to make a PR adding this to the template, I can style it.
> I assume we can simply output this information on the provisioning page? If you want to make a PR adding this to the template, I can style it.

Yep, should be as simple as that. I'll open a PR in a bit.

Edit: An idea: we might want to expose this as a clipboard action. @nlhkabu would that be possible?

Resolving patch:
diff --git a/warehouse/manage/views.py b/warehouse/manage/views.py
--- a/warehouse/manage/views.py
+++ b/warehouse/manage/views.py
@@ -10,6 +10,7 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 
+import base64
 import io
 
 from collections import defaultdict
@@ -323,6 +324,7 @@ def __init__(self, request):
     def default_response(self):
         totp_secret = self.request.session.get_totp_secret()
         return {
+            "provision_totp_secret": base64.b32encode(totp_secret).decode(),
             "provision_totp_form": ProvisionTOTPForm(totp_secret=totp_secret),
             "provision_totp_uri": otp.generate_totp_provisioning_uri(
                 totp_secret,


Test patch:
diff --git a/tests/unit/manage/test_views.py b/tests/unit/manage/test_views.py
--- a/tests/unit/manage/test_views.py
+++ b/tests/unit/manage/test_views.py
@@ -10,6 +10,7 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 
+import base64
 import datetime
 import uuid
 
@@ -782,6 +783,7 @@ def test_totp_provision(self, monkeypatch):
 
         assert provision_totp_cls.calls == [pretend.call(totp_secret=b"secret")]
         assert result == {
+            "provision_totp_secret": base64.b32encode(b"secret").decode(),
             "provision_totp_form": provision_totp_obj,
             "provision_totp_uri": "not_a_real_uri",
         }
@@ -912,7 +914,7 @@ def test_validate_totp_provision_invalid_form(self, monkeypatch):
             POST={},
             session=pretend.stub(
                 flash=pretend.call_recorder(lambda *a, **kw: None),
-                get_totp_secret=lambda: pretend.stub(),
+                get_totp_secret=lambda: b"secret",
             ),
             find_service=lambda *a, **kw: user_service,
             user=pretend.stub(
@@ -943,6 +945,7 @@ def test_validate_totp_provision_invalid_form(self, monkeypatch):
 
         assert request.session.flash.calls == []
         assert result == {
+            "provision_totp_secret": base64.b32encode(b"secret").decode(),
             "provision_totp_form": provision_totp_obj,
             "provision_totp_uri": "not_a_real_uri",
         }
Repository: pypi/warehouse

Issue:
Improve webauthn errors
Problems:

- We currently have two pieces of JS that control the display of webauthn errors - some in `index.js`, some in `webauthn.js`
- The errors are not announced to the screenreader (via `role=alert`)
- The errors are not associated with the webauthn label field (on the provisioning page) - we should use `aria-describedby` for this
- The user is able to put text into the label field on the provisioning page - it should be disabled


Discussion:
@woodruffw as most of this JS is yours, it would be helpful if we could look into this together?  Thanks
Yep! Let's set aside some time for this before ending the WebAuthn beta.

Resolving patch:
diff --git a/warehouse/manage/forms.py b/warehouse/manage/forms.py
--- a/warehouse/manage/forms.py
+++ b/warehouse/manage/forms.py
@@ -111,11 +111,11 @@ def validate_totp_value(self, field):
 
 
 class DeleteWebAuthnForm(forms.Form):
-    __params__ = ["confirm_key_name"]
+    __params__ = ["confirm_device_name"]
 
     label = wtforms.StringField(
         validators=[
-            wtforms.validators.DataRequired(message="Specify a label"),
+            wtforms.validators.DataRequired(message="Specify a device name"),
             wtforms.validators.Length(
                 max=64, message=("Label must be 64 characters or less")
             ),


Test patch:
diff --git a/tests/unit/manage/test_forms.py b/tests/unit/manage/test_forms.py
--- a/tests/unit/manage/test_forms.py
+++ b/tests/unit/manage/test_forms.py
@@ -293,7 +293,7 @@ def test_validate_label_missing(self):
         )
 
         assert not form.validate()
-        assert form.label.errors.pop() == "Specify a label"
+        assert form.label.errors.pop() == "Specify a device name"
 
     def test_validate_label_not_in_use(self):
         user_service = pretend.stub(
Repository: pypi/warehouse

Issue:
PyPI accepts packages with dependencies on local versions (e.g., 0.1.0+local)
PyPI accepts packages with dependencies on local versions (e.g., 0.1.0+local). I'm not sure if this is intentional or not, since PyPI will reject packages whose version is a local version.

I tested this was the case using this test package:

```
import setuptools
import os
import re

setuptools.setup(
    name="test-pypi-version-specifier-main",
    version="0.0.2",
    author="Edward Z. Yang",
    author_email="ezyang@mit.edu",
    description="Testing package",
    long_description="Yarr",
    long_description_content_type="text/markdown",
    url="https://localhost/",
    packages=setuptools.find_packages(),
    include_package_data=True,
    classifiers=[
        "Programming Language :: Python :: 3",
        "License :: OSI Approved :: MIT License",
        "Operating System :: OS Independent",
    ],
    install_requires=[
        "test-pypi-version-specifier-dep==0.0.1+cuda9",
    ],
)

```


Discussion:

Resolving patch:
diff --git a/warehouse/forklift/legacy.py b/warehouse/forklift/legacy.py
--- a/warehouse/forklift/legacy.py
+++ b/warehouse/forklift/legacy.py
@@ -252,6 +252,11 @@ def _validate_legacy_dist_req(requirement):
             "Can't have direct dependency: {!r}".format(requirement)
         )
 
+    if any(packaging.version.Version(spec.version).local for spec in req.specifier):
+        raise wtforms.validators.ValidationError(
+            "Can't have dependency with local version: {!r}".format(requirement)
+        )
+
 
 def _validate_legacy_dist_req_list(form, field):
     for datum in field.data:


Test patch:
diff --git a/tests/unit/forklift/test_legacy.py b/tests/unit/forklift/test_legacy.py
--- a/tests/unit/forklift/test_legacy.py
+++ b/tests/unit/forklift/test_legacy.py
@@ -170,6 +170,7 @@ def test_validate_legacy_dist_req_valid(self, requirement):
             "_foo",
             "_foo (>=1.0)",
             "name @ https://github.com/pypa",
+            "test-pypi-version-specifier-dep==0.0.1+cuda9",
         ],
     )
     def test_validate_legacy_dist_req_invalid(self, requirement):
Repository: pypi/warehouse

Issue:
warehouse: Change default scope from disabled to hidden
Prevents empty value submission, which in turn causes a routing
match failure, which in turn causes a 404 instead of a form error.

cc @di 


Discussion:

Resolving patch:
diff --git a/warehouse/manage/views.py b/warehouse/manage/views.py
--- a/warehouse/manage/views.py
+++ b/warehouse/manage/views.py
@@ -579,7 +579,7 @@ def default_response(self):
     def manage_macaroons(self):
         return self.default_response
 
-    @view_config(request_method="POST", request_param=CreateMacaroonForm.__params__)
+    @view_config(request_method="POST")
     def create_macaroon(self):
         if not self.request.user.has_primary_verified_email:
             self.request.session.flash(


Test patch:
Repository: pypi/warehouse

Issue:
Upload to PyPI fails when dependency version contains *
**Describe the bug**
PyPI raises `400 Client error` when uploading a package that specifies requirement using `== 2.*`. This is a valid version specifier referred to [PEP 440](https://www.python.org/dev/peps/pep-0440/#compatible-release).

The whole error is:
```
HTTPError: 400 Client Error: Invalid value for requires_dist. Error: Invalid version: '2.*' for url: https://test.pypi.org/legacy/
```

**Expected behavior**
Upload should pass with no errors. (Note that in the example below the expected behaviour would be to fail with authentication error, as you don't have sufficient permissions on the project.)

**To Reproduce**
This is a minimal reproducer: https://github.com/dblenkus/warehouse-requirements-issue

Install twine with `pip install twine` and try to upload the package with `twine upload -r testpypi dist/*`.

**My Platform**
MacOS 10.14.6
twine 1.13.0

**Additional context**
This has worked few days ago on July 26th 2019.


Discussion:

Resolving patch:
diff --git a/warehouse/forklift/legacy.py b/warehouse/forklift/legacy.py
--- a/warehouse/forklift/legacy.py
+++ b/warehouse/forklift/legacy.py
@@ -252,11 +252,6 @@ def _validate_legacy_dist_req(requirement):
             "Can't have direct dependency: {!r}".format(requirement)
         )
 
-    if any(packaging.version.Version(spec.version).local for spec in req.specifier):
-        raise wtforms.validators.ValidationError(
-            "Can't have dependency with local version: {!r}".format(requirement)
-        )
-
 
 def _validate_legacy_dist_req_list(form, field):
     for datum in field.data:


Test patch:
diff --git a/tests/unit/forklift/test_legacy.py b/tests/unit/forklift/test_legacy.py
--- a/tests/unit/forklift/test_legacy.py
+++ b/tests/unit/forklift/test_legacy.py
@@ -155,7 +155,8 @@ def test_validate_legacy_non_dist_req_list(self, monkeypatch):
         assert validator.calls == [pretend.call(datum) for datum in data]
 
     @pytest.mark.parametrize(
-        "requirement", ["foo (>=1.0)", "foo", "foo2", "foo-bar", "foo_bar"]
+        "requirement",
+        ["foo (>=1.0)", "foo", "foo2", "foo-bar", "foo_bar", "foo == 2.*"],
     )
     def test_validate_legacy_dist_req_valid(self, requirement):
         legacy._validate_legacy_dist_req(requirement)
@@ -170,7 +171,6 @@ def test_validate_legacy_dist_req_valid(self, requirement):
             "_foo",
             "_foo (>=1.0)",
             "name @ https://github.com/pypa",
-            "test-pypi-version-specifier-dep==0.0.1+cuda9",
         ],
     )
     def test_validate_legacy_dist_req_invalid(self, requirement):
Repository: pypi/warehouse

Issue:
Invalid HTML for select element
This html is generated by the Python form code.

template:
https://github.com/pypa/warehouse/blob/master/warehouse/templates/manage/roles.html

field:
`{{ form.role_name }}`

ERROR: The first child “option” element of a “select” element with a “required” attribute, and without a “multiple” attribute, and without a “size” attribute whose value is greater than “1”, must have either an empty “value” attribute, or must have no text content. Consider either adding a placeholder option label, or adding a “size” attribute with a value equal to the number of “option” elements. (433)

Reference:
https://maxdesign.com.au/articles/select-required/



Discussion:
I'm having a bit of trouble identifying this one: I see one `<select>` in that template:

```html
<select class="change-field" name="role_name" data-original="{{ role.role_name }}">
```

But it doesn't have a `required` attribute or another of the other attributes that would constitute the error, unless I'm missing something.
@woodruffw I think it's here:
https://github.com/pypa/warehouse/blob/master/warehouse/templates/manage/roles.html#L138

The `<select>` isn't present in the template, it's being pulled in by `{{ form.role_name }}`. Therefore the required attribute must be defined in the backend.

If I visit `http://localhost/manage/project/pip/collaboration/`, I get the following HTML when inspecting the 'role' select on the form where users can add a collaborator:

![Screenshot from 2019-08-13 19-23-18](https://user-images.githubusercontent.com/3323703/62967160-52cdb080-be00-11e9-86e9-f0d5d6d32514.png)

![Screenshot from 2019-08-13 19-23-39](https://user-images.githubusercontent.com/3323703/62967159-52351a00-be00-11e9-8c34-c20326dc32c3.png)
Hmm, this might actually be a problem with WTForms' HTML generation. Here's the underlying form mixin:

```python
class RoleNameMixin:

    role_name = wtforms.SelectField(
        "Select role",
        choices=[("Maintainer", "Maintainer"), ("Owner", "Owner")],
        validators=[wtforms.validators.DataRequired(message="Select role")],
    )
```

So the fix here is probably upstream -- `choices` (or the underlying generator that uses it, since it can be dynamic) needs to properly set the `size` attribute. Alternatively, we could add an additional `("", "Select role")` choice, although I'm not sure what that does at the validation level. I'll try it out.

Resolving patch:
diff --git a/warehouse/manage/forms.py b/warehouse/manage/forms.py
--- a/warehouse/manage/forms.py
+++ b/warehouse/manage/forms.py
@@ -31,7 +31,7 @@ class RoleNameMixin:
 
     role_name = wtforms.SelectField(
         "Select role",
-        choices=[("Maintainer", "Maintainer"), ("Owner", "Owner")],
+        choices=[("", "Select role"), ("Maintainer", "Maintainer"), ("Owner", "Owner")],
         validators=[wtforms.validators.DataRequired(message="Select role")],
     )
 


Test patch:
Repository: pypi/warehouse

Issue:
500 error on project deletion
Currently project deletion results in a 500 error due to #6339.

This is because the `UserEvent` and `ProjectEvent` models have a 'not-null' constraint on the project ID, and are not cascaded (deleted) when the project is deleted.


Discussion:

Resolving patch:
diff --git a/warehouse/accounts/models.py b/warehouse/accounts/models.py
--- a/warehouse/accounts/models.py
+++ b/warehouse/accounts/models.py
@@ -96,7 +96,9 @@ class User(SitemapMixin, db.Model):
         "Macaroon", backref="user", cascade="all, delete-orphan", lazy=False
     )
 
-    events = orm.relationship("UserEvent", backref="user", lazy=False)
+    events = orm.relationship(
+        "UserEvent", backref="user", cascade="all, delete-orphan", lazy=False
+    )
 
     def record_event(self, *, tag, ip_address, additional):
         session = orm.object_session(self)
diff --git a/warehouse/packaging/models.py b/warehouse/packaging/models.py
--- a/warehouse/packaging/models.py
+++ b/warehouse/packaging/models.py
@@ -136,7 +136,9 @@ class Project(SitemapMixin, db.Model):
         passive_deletes=True,
     )
 
-    events = orm.relationship("ProjectEvent", backref="project", lazy=False)
+    events = orm.relationship(
+        "ProjectEvent", backref="project", cascade="all, delete-orphan", lazy=False
+    )
 
     def __getitem__(self, version):
         session = orm.object_session(self)


Test patch:
Repository: pypi/warehouse

Issue:
Add sitemap, link to in footer
WCAG 2.4.5:

> Multiple ways are available to find other web pages on the site - at least two of: a list of
related pages, table of contents, site map, site search, or list of all available web pages.

The quickest way for us to meet this criteria is to add a basic sitemap to PyPI and link to it from the site footer.


Discussion:
@woodruffw - I've assigned this one to you as some backend work will be required here.
Sitemap is here: https://pypi.org/sitemap.xml
Thanks @di - but I don't know that this would actually be useful/usable for people wanting a secondary method of navigation.

The recommendations say:
> The simplest and most common kind of site map is an outline that shows links to each section or sub-site. Such outline views do not show more complex relationships within the site, such as links between pages in different sections of the site. The site maps for some large sites use headings that expand to show additional detail about each section.

Further guidance on how to create site map:
https://www.w3.org/TR/2016/NOTE-WCAG20-TECHS-20161007/G63

Example site map:
https://www.w3.org/WAI/sitemap/

Sorry, should have added more context! Wanted to point y'all to the existing machine-readable sitemap as the view could be reused for a human-readable sitemap (i.e., we don't need to start from scratch).

Resolving patch:
diff --git a/warehouse/routes.py b/warehouse/routes.py
--- a/warehouse/routes.py
+++ b/warehouse/routes.py
@@ -33,6 +33,7 @@ def includeme(config):
     config.add_route("bucket.sitemap.xml", "/{bucket}.sitemap.xml", domain=warehouse)
 
     # Some static, template driven pages
+    config.add_template_view("sitemap", "/sitemap/", "pages/sitemap.html")
     config.add_template_view("help", "/help/", "pages/help.html")
     config.add_template_view("security", "/security/", "pages/security.html")
     config.add_template_view(


Test patch:
diff --git a/tests/unit/test_routes.py b/tests/unit/test_routes.py
--- a/tests/unit/test_routes.py
+++ b/tests/unit/test_routes.py
@@ -327,6 +327,7 @@ def add_policy(name, filename):
     ]
 
     assert config.add_template_view.calls == [
+        pretend.call("sitemap", "/sitemap/", "pages/sitemap.html"),
         pretend.call("help", "/help/", "pages/help.html"),
         pretend.call("security", "/security/", "pages/security.html"),
         pretend.call(
Repository: pypi/warehouse

Issue:
Translation links in footer should display in local language
For example:
- "French" should be "Français"
- "Spanish" should be "Español"

Etc.


Discussion:

Resolving patch:
diff --git a/warehouse/i18n/__init__.py b/warehouse/i18n/__init__.py
--- a/warehouse/i18n/__init__.py
+++ b/warehouse/i18n/__init__.py
@@ -19,13 +19,15 @@
 
 from warehouse.cache.http import add_vary
 
+# Taken from:
+# https://github.com/django/django/blob/master/django/conf/locale/__init__.py
 KNOWN_LOCALES = {
     "en": "English",
-    "es": "Spanish",
-    "fr": "French",
-    "ja": "Japanese",
-    "pt_BR": "Portuguese (Brazil)",
-    "uk": "Ukrainian",
+    "es": "español",
+    "fr": "français",
+    "ja": "日本語",
+    "pt_BR": "Português Brasileiro",
+    "uk": "Українська",
 }
 
 LOCALE_ATTR = "_LOCALE_"


Test patch:
Repository: pypi/warehouse

Issue:
WebAuthN user verification flag should be set to discouraged
The current setting on the Pypi site is to ignore the UserVerificationOption setting in the `navigator.credentials.create()` call. Based on [this chromium documentation](https://chromium.googlesource.com/chromium/src/+/master/content/browser/webauth/uv_preferred.md), it appears to be set to "preferred" by default.

TL:DR; it interrupts the login flow by popping up a pin entry dialog upon tapping my yubikey, then requires me to tap again after successful pin entry. This is not the recommended setting for second-factor credentials, only for passwordless [[1](https://developers.yubico.com/WebAuthn/WebAuthn_Developer_Guide/User_Presence_vs_User_Verification.html)].

Please explicitly set this to [discouraged](https://w3c.github.io/webauthn/#enum-userVerificationRequirement) instead, so that I will no longer have PIN prompts on second-factor login.


Discussion:
Thanks for filing this issue and providing great references, @gcochard!

I was able to reproduce this using a FIDO2 compatible token and Google Chrome on MacOS.

Based on the your provided references it seems like your suggestion is correct.

The library we use for Webauthn on the server has implemented this option in https://github.com/duo-labs/py_webauthn/commit/44bb8241ca6a10b6d50f20f32452500ddebecb9d but has not created a release since.

A release has already been requested for the same reason in https://github.com/duo-labs/py_webauthn/issues/60.

Once we get an update to the webauthn library this shouldn't be to bad to see implemented.
v0.4.6 has been released!

Resolving patch:
diff --git a/warehouse/utils/webauthn.py b/warehouse/utils/webauthn.py
--- a/warehouse/utils/webauthn.py
+++ b/warehouse/utils/webauthn.py
@@ -80,7 +80,14 @@ def get_credential_options(user, *, challenge, rp_name, rp_id):
     on the client side.
     """
     options = pywebauthn.WebAuthnMakeCredentialOptions(
-        challenge, rp_name, rp_id, str(user.id), user.username, user.name, None
+        challenge,
+        rp_name,
+        rp_id,
+        str(user.id),
+        user.username,
+        user.name,
+        None,
+        user_verification="discouraged",
     )
 
     return options.registration_dict


Test patch:
Repository: pypi/warehouse

Issue:
Fix simple typo: prinicpal -> principal
# Issue Type

[x] Bug (Typo)

# Steps to Replicate

1. Examine warehouse/macaroons/auth_policy.py.
2. Search for `prinicpal`.

# Expected Behaviour

1. Should read `principal`.




Discussion:

Resolving patch:
diff --git a/warehouse/macaroons/auth_policy.py b/warehouse/macaroons/auth_policy.py
--- a/warehouse/macaroons/auth_policy.py
+++ b/warehouse/macaroons/auth_policy.py
@@ -144,7 +144,7 @@ def permits(self, context, principals, permission):
 
             # If our Macaroon is verified, and for a valid permission then we'll pass
             # this request to our underlying Authorization policy, so it can handle its
-            # own authorization logic on the prinicpal.
+            # own authorization logic on the principal.
             if permission in valid_permissions:
                 return self.policy.permits(context, principals, permission)
             else:


Test patch:
Repository: pypi/warehouse

Issue:
Fix simple typo: thorugh -> through
# Issue Type

[x] Bug (Typo)

# Steps to Replicate

1. Examine warehouse/filters.py.
2. Search for `thorugh`.

# Expected Behaviour

1. Should read `through`.




Discussion:

Resolving patch:
diff --git a/warehouse/filters.py b/warehouse/filters.py
--- a/warehouse/filters.py
+++ b/warehouse/filters.py
@@ -132,7 +132,7 @@ def format_classifiers(classifiers):
         if value:
             structured[key].append(value[0])
 
-    # Go thorugh and ensure that all of the lists in our classifiers are in
+    # Go through and ensure that all of the lists in our classifiers are in
     # sorted order.
     structured = {k: sorted(v) for k, v in structured.items()}
 


Test patch: