internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: pypi/warehouse

Issue:
Refuse account creation when using disposable email addresses.
long term it makes no sense to accept accounts which use an email address which is disposable for managing legit packages. short/near term it opens an easy door for spammers to create accounts on PyPI.

i've implemented blacklisting for account signup and email swaps which use the blacklist at https://github.com/martenson/disposable-email-domains for legacy pypi.



Discussion:

Resolving patch:
diff --git a/warehouse/accounts/forms.py b/warehouse/accounts/forms.py
--- a/warehouse/accounts/forms.py
+++ b/warehouse/accounts/forms.py
@@ -11,6 +11,7 @@
 # limitations under the License.
 import re
 
+import disposable_email_domains
 import wtforms
 import wtforms.fields.html5
 
@@ -84,6 +85,9 @@ def validate_username(self, field):
     def validate_email(self, field):
         if self.user_service.find_userid_by_email(field.data) is not None:
             raise wtforms.validators.ValidationError("Email exists.")
+        domain = field.data.split('@')[-1]
+        if domain in disposable_email_domains.blacklist:
+            raise wtforms.validators.ValidationError("Disposable email.")
 
     def validate_g_recaptcha_response(self, field):
         # do required data validation here due to enabled flag being required


Test patch:
diff --git a/tests/unit/accounts/test_forms.py b/tests/unit/accounts/test_forms.py
--- a/tests/unit/accounts/test_forms.py
+++ b/tests/unit/accounts/test_forms.py
@@ -204,6 +204,18 @@ def test_email_exists_error(self):
         assert not form.validate()
         assert form.email.errors.pop() == "Email exists."
 
+    def test_blacklisted_email_error(self):
+        form = forms.RegistrationForm(
+            data={"email": "foo@bearsarefuzzy.com"},
+            user_service=pretend.stub(
+                find_userid_by_email=pretend.call_recorder(lambda _: None),
+            ),
+            recaptcha_service=pretend.stub(enabled=True),
+        )
+
+        assert not form.validate()
+        assert form.email.errors.pop() == "Disposable email."
+
     def test_recaptcha_disabled(self):
         form = forms.RegistrationForm(
             data={"g_recpatcha_response": ""},
Repository: pypi/warehouse

Issue:
[requires.io] dependency update on master branch



Discussion:

Resolving patch:
diff --git a/warehouse/celery.py b/warehouse/celery.py
--- a/warehouse/celery.py
+++ b/warehouse/celery.py
@@ -10,11 +10,11 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 
-import celery.backends
+import celery.app.backends
 
 # We need to trick Celery into supporting rediss:// URLs which is how redis-py
 # signals that you should use Redis with TLS.
-celery.backends.BACKEND_ALIASES["rediss"] = "warehouse.celery:TLSRedisBackend"  # noqa
+celery.app.backends.BACKEND_ALIASES["rediss"] = "warehouse.celery:TLSRedisBackend"  # noqa
 
 from celery import Celery, Task
 from celery.backends.redis import RedisBackend as _RedisBackend


Test patch:
Repository: pypi/warehouse

Issue:
json urls object may contain prereleases
I think the urls object in the JSON output should reference downloads for the latest stable release; even after #1519, the urls object may contain references to prerelease versions. For example, https://pypi.org/pypi/wheel/json has

```json
{
  "urls": [
    {
      "filename": "wheel-0.30.0a0-py2.py3-none-any.whl",
      "url": "https://files.pythonhosted.org/packages/83/53/e120833aa2350db333df89a40dea3b310dd9dabf6f29eaa18934a597dc79/wheel-0.30.0a0-py2.py3-none-any.whl"
    }, ...
```

I think it should point to 0.29.0 instead, like the project page does.


Discussion:

Resolving patch:
diff --git a/warehouse/legacy/api/json.py b/warehouse/legacy/api/json.py
--- a/warehouse/legacy/api/json.py
+++ b/warehouse/legacy/api/json.py
@@ -42,7 +42,9 @@ def json_project(project, request):
         release = (
             request.db.query(Release)
                       .filter(Release.project == project)
-                      .order_by(Release._pypi_ordering.desc())
+                      .order_by(
+                            Release.is_prerelease.nullslast(),
+                            Release._pypi_ordering.desc())
                       .limit(1)
                       .one()
         )


Test patch:
diff --git a/tests/unit/legacy/api/test_json.py b/tests/unit/legacy/api/test_json.py
--- a/tests/unit/legacy/api/test_json.py
+++ b/tests/unit/legacy/api/test_json.py
@@ -66,6 +66,41 @@ def test_calls_release_detail(self, monkeypatch, db_request):
         assert resp is response
         assert json_release.calls == [pretend.call(release, db_request)]
 
+    def test_with_prereleases(self, monkeypatch, db_request):
+        project = ProjectFactory.create()
+
+        ReleaseFactory.create(project=project, version="1.0")
+        ReleaseFactory.create(project=project, version="2.0")
+        ReleaseFactory.create(project=project, version="4.0.dev0")
+
+        release = ReleaseFactory.create(project=project, version="3.0")
+
+        response = pretend.stub()
+        json_release = pretend.call_recorder(lambda ctx, request: response)
+        monkeypatch.setattr(json, "json_release", json_release)
+
+        resp = json.json_project(project, db_request)
+
+        assert resp is response
+        assert json_release.calls == [pretend.call(release, db_request)]
+
+    def test_only_prereleases(self, monkeypatch, db_request):
+        project = ProjectFactory.create()
+
+        ReleaseFactory.create(project=project, version="1.0.dev0")
+        ReleaseFactory.create(project=project, version="2.0.dev0")
+
+        release = ReleaseFactory.create(project=project, version="3.0.dev0")
+
+        response = pretend.stub()
+        json_release = pretend.call_recorder(lambda ctx, request: response)
+        monkeypatch.setattr(json, "json_release", json_release)
+
+        resp = json.json_project(project, db_request)
+
+        assert resp is response
+        assert json_release.calls == [pretend.call(release, db_request)]
+
 
 class TestJSONRelease:
Repository: pypi/warehouse

Issue:
Update elasticsearch-dsl to 5.1.0

There's a new version of [elasticsearch-dsl](https://pypi.python.org/pypi/elasticsearch-dsl) available.
You are currently using **5.0.0**. I have updated it to **5.1.0**



These links might come in handy:  <a href="http://pypi.python.org/pypi/elasticsearch-dsl">PyPI</a> | <a href="https://pyup.io/changelogs/elasticsearch-dsl/">Changelog</a> | <a href="https://github.com/elasticsearch/elasticsearch-dsl-py">Repo</a> 



### Changelog
> 
>### 5.1.0

>------------------
> * Renamed ``Result`` and ``ResultMeta`` to ``Hit`` and ``HitMeta`` respectively
> * ``Response`` now stores ``Search`` which it gets as first arg to ``__init__``
> * aggregation results are now wrapped in classes and properly deserialized
> * ``Date`` fields now allow for numerical timestamps in the java format (in millis)
> * Added API documentation
> * replaced generated classes with manually created







*Got merge conflicts? Close this PR and delete the branch. I'll create a new PR for you.*

Happy merging! 🤖



Discussion:

Resolving patch:
diff --git a/warehouse/search.py b/warehouse/search.py
--- a/warehouse/search.py
+++ b/warehouse/search.py
@@ -16,7 +16,7 @@
 import elasticsearch
 import venusian
 
-from elasticsearch_dsl import Index
+from elasticsearch_dsl import Index, serializer
 
 
 def doc_type(cls):
@@ -64,6 +64,7 @@ def includeme(config):
         ca_certs=certifi.where(),
         timeout=30,
         retry_on_timeout=True,
+        serializer=serializer.serializer,
     )
     config.registry["elasticsearch.index"] = p.path.strip("/")
     config.registry["elasticsearch.shards"] = int(qs.get("shards", ["1"])[0])


Test patch:
Repository: pypi/warehouse

Issue:
"Newer version available" not correct if pre-release page
No big deal, but after the release of #1519, when you are on a *recent* preview package the version badge shows "Newer version available" for the latest stable:

![image](https://cloud.githubusercontent.com/assets/1050156/20896960/efa29c9a-bad4-11e6-8a34-40426930a5cd.png)

This is technically false, this is not a newer version, but the latest stable. I humbly think in this case this might confuse people.

This might be true for a 1.0.0rc1 compared to a 1.0.0, but not for a 2.0.0rc1 compared to a 1.0.0.

FYI @brettcannon


Discussion:
Ah yes. It should maybe be orange or something that just says that this is a pre-release version and the latest stable is X.

Resolving patch:
diff --git a/warehouse/config.py b/warehouse/config.py
--- a/warehouse/config.py
+++ b/warehouse/config.py
@@ -262,6 +262,7 @@ def configure(settings=None):
         "format_package_type",
         "warehouse.filters:format_package_type"
     )
+    filters.setdefault("parse_version", "warehouse.filters:parse_version")
 
     # We also want to register some global functions for Jinja
     jglobals = config.get_settings().setdefault("jinja2.globals", {})
diff --git a/warehouse/filters.py b/warehouse/filters.py
--- a/warehouse/filters.py
+++ b/warehouse/filters.py
@@ -23,6 +23,7 @@
 import html5lib.treewalkers
 import jinja2
 
+import packaging.version
 import readme_renderer.rst
 import readme_renderer.txt
 
@@ -164,3 +165,7 @@ def contains_valid_uris(items):
     URIs
     """
     return any(is_valid_uri(i) for i in items)
+
+
+def parse_version(version_str):
+    return packaging.version.parse(version_str)


Test patch:
diff --git a/tests/functional/test_templates.py b/tests/functional/test_templates.py
--- a/tests/functional/test_templates.py
+++ b/tests/functional/test_templates.py
@@ -42,6 +42,7 @@ def test_templates_for_empty_titles():
         "urlparse": "warehouse.filters:urlparse",
         "contains_valid_uris": "warehouse.filters:contains_valid_uris",
         "format_package_type": "warehouse.filters:format_package_type",
+        "parse_version": "warehouse.filters:parse_version",
     })
 
     for dir_, _, files in os.walk(dir_name):
diff --git a/tests/unit/test_filters.py b/tests/unit/test_filters.py
--- a/tests/unit/test_filters.py
+++ b/tests/unit/test_filters.py
@@ -13,6 +13,7 @@
 import urllib.parse
 
 import jinja2
+import packaging.version
 import pretend
 import pytest
 import readme_renderer.rst
@@ -246,3 +247,13 @@ def test_contains_valid_uris(inp, expected):
 )
 def test_format_package_type(inp, expected):
     assert filters.format_package_type(inp) == expected
+
+
+@pytest.mark.parametrize(
+    ("inp", "expected"),
+    [
+        ("1.0", packaging.version.Version("1.0")),
+    ]
+)
+def test_parse_version(inp, expected):
+    assert filters.parse_version(inp) == expected
Repository: pypi/warehouse

Issue:
Pagination for releases on Project Admin doesn't work
The pagination on releases does not appear to be functional in the project admin. It shows the first N releases, but when you click to see all, there is no pagination links and there is no text to indicate what page you're on or how many results there are.

Manually adding a ``?page=2`` *does* work, so this is likely just something wrong in the template.


Discussion:
@clgarciga hi!
references #2374 

Resolving patch:
diff --git a/warehouse/admin/views/projects.py b/warehouse/admin/views/projects.py
--- a/warehouse/admin/views/projects.py
+++ b/warehouse/admin/views/projects.py
@@ -149,7 +149,7 @@ def releases_list(project, request):
     )
 
     return {
-        "releases": list(releases),
+        "releases": releases,
         "project": project,
         "query": q,
     }


Test patch:
Repository: pypi/warehouse

Issue:
Link to 'edit project' when logged in user is owner
I have added a button in the UI to link *back* the the manage project page, when the user is viewing their project.

![screenshot from 2018-01-18 07-25-07](https://user-images.githubusercontent.com/3323703/35085471-ca4590a4-fc20-11e7-8f2e-4800b119b9fe.png)

This is currently commented out.



Discussion:
A sub ticket of #424
This will need it's own client-side-include route.

Resolving patch:
diff --git a/warehouse/packaging/views.py b/warehouse/packaging/views.py
--- a/warehouse/packaging/views.py
+++ b/warehouse/packaging/views.py
@@ -126,3 +126,13 @@ def release_detail(release, request):
         "maintainers": maintainers,
         "license": license,
     }
+
+
+@view_config(
+    route_name="includes.edit-project-button",
+    renderer="includes/edit-project-button.html",
+    uses_session=True,
+    permission="manage",
+)
+def edit_project_button(project, request):
+    return {'project': project}
diff --git a/warehouse/routes.py b/warehouse/routes.py
--- a/warehouse/routes.py
+++ b/warehouse/routes.py
@@ -68,6 +68,13 @@ def includeme(config):
         traverse="/{username}",
         domain=warehouse,
     )
+    config.add_route(
+        "includes.edit-project-button",
+        "/_includes/edit-project-button/{project_name}",
+        factory="warehouse.packaging.models:ProjectFactory",
+        traverse="/{project_name}",
+        domain=warehouse,
+    )
 
     # Search Routes
     config.add_route("search", "/search/", domain=warehouse)


Test patch:
diff --git a/tests/unit/packaging/test_views.py b/tests/unit/packaging/test_views.py
--- a/tests/unit/packaging/test_views.py
+++ b/tests/unit/packaging/test_views.py
@@ -231,3 +231,12 @@ def test_multiple_licenses_from_classifiers(self, db_request):
         result = views.release_detail(release, db_request)
 
         assert result["license"] == "BSD License, MIT License"
+
+
+class TestEditProjectButton:
+
+    def test_edit_project_button_returns_project(self):
+        project = pretend.stub()
+        assert views.edit_project_button(project, pretend.stub()) == {
+            'project': project
+        }
diff --git a/tests/unit/test_routes.py b/tests/unit/test_routes.py
--- a/tests/unit/test_routes.py
+++ b/tests/unit/test_routes.py
@@ -100,6 +100,13 @@ def add_policy(name, filename):
             traverse="/{username}",
             domain=warehouse,
         ),
+        pretend.call(
+            "includes.edit-project-button",
+            "/_includes/edit-project-button/{project_name}",
+            factory="warehouse.packaging.models:ProjectFactory",
+            traverse="/{project_name}",
+            domain=warehouse,
+        ),
         pretend.call("search", "/search/", domain=warehouse),
         pretend.call(
             "accounts.profile",
Repository: pypi/warehouse

Issue:
duplicate footer appears in package page only when not logged in
When viewing a package page (appears to apply to any package), the (blue) footer ("Get Help", "About PyPI", "Contributing to PyPI") occurs twice: once below a login prompt and a search text box that appear below the package's title (with the package details below these and the footer), and once at the bottom of the page (the latter as expected).

This behavior occurs only when logged out. In contrast, when logged in, the layout seems as intended: a search text box at the top, and the package title and details contiguously below that search box, with a single footer at the bottom.


Discussion:
<img width="565" alt="screen shot 2018-01-26 at 7 05 03 pm" src="https://user-images.githubusercontent.com/3275593/35442066-dc315d82-02cb-11e8-8eeb-d879a8181cab.png">

Indeed, this is how it looks, with the bottom footer appearing after the section of details that starts with "Navigation". Thanks for adding a picture.
Going to guess an HTML include got messed up, i think @di added one here recently?
Yeah, I know what's causing this. On it.

Resolving patch:
diff --git a/warehouse/views.py b/warehouse/views.py
--- a/warehouse/views.py
+++ b/warehouse/views.py
@@ -125,6 +125,14 @@ def forbidden(exc, request, redirect_to="accounts.login"):
     return httpexception_view(exc, request)
 
 
+@forbidden_view_config(path_info=r"^/_includes/")
+@exception_view_config(PredicateMismatch, path_info=r"^/_includes/")
+def forbidden_include(exc, request):
+    # If the forbidden error is for a client-side-include, just return an empty
+    # response instead of redirecting
+    return Response()
+
+
 @view_config(
     route_name="robots.txt",
     renderer="robots.txt",


Test patch:
diff --git a/tests/unit/test_views.py b/tests/unit/test_views.py
--- a/tests/unit/test_views.py
+++ b/tests/unit/test_views.py
@@ -25,7 +25,7 @@
 from warehouse.views import (
     SEARCH_BOOSTS, SEARCH_FIELDS, current_user_indicator, forbidden, health,
     httpexception_view, index, robotstxt, opensearchxml, search, force_status,
-    flash_messages
+    flash_messages, forbidden_include
 )
 
 from ..common.db.accounts import UserFactory
@@ -147,6 +147,19 @@ def test_logged_out_redirects_login(self):
             "/accounts/login/?next=/foo/bar/%3Fb%3Ds"
 
 
+class TestForbiddenIncludeView:
+
+    def test_forbidden_include(self):
+        exc = pretend.stub()
+        request = pretend.stub()
+
+        resp = forbidden_include(exc, request)
+
+        assert resp.status_code == 200
+        assert resp.content_type == 'text/html'
+        assert resp.content_length == 0
+
+
 def test_robotstxt(pyramid_request):
     assert robotstxt(pyramid_request) == {}
     assert pyramid_request.response.content_type == "text/plain"
Repository: pypi/warehouse

Issue:
Blacklisting project does not purge the cache



Discussion:
In our [meeting today](https://wiki.python.org/psf/PackagingWG/2017-12-19-Warehouse) we decided this belongs in [the first milestone, the MVP for package maintainers](https://github.com/pypa/warehouse/milestone/8). Best wishes and I wish you happy and easy bug-fixing, Ernest!

Resolving patch:
diff --git a/warehouse/admin/utils.py b/warehouse/admin/utils.py
--- a/warehouse/admin/utils.py
+++ b/warehouse/admin/utils.py
@@ -64,8 +64,22 @@ def remove_project(project, request):
     (request.db.execute(release_classifiers.delete()
                         .where(release_classifiers.c.name ==
                                project.name)))
-    request.db.query(Release).filter(Release.name == project.name).delete()
-    request.db.query(Project).filter(Project.name == project.name).delete()
+
+    # Load the following objects into the session and individually delete them
+    # so they are included in `session.deleted` and their cache keys are purged
+
+    # Delete releases first, otherwise they will get cascade-deleted by the
+    # project deletion and won't be purged
+    for release in (
+            request.db.query(Release)
+            .filter(Release.name == project.name)
+            .all()):
+        request.db.delete(release)
+
+    # Finally, delete the project
+    request.db.delete(
+        request.db.query(Project).filter(Project.name == project.name).one()
+    )
 
     request.session.flash(
         f"Successfully deleted the project {project.name!r}.",


Test patch:
Repository: pypi/warehouse

Issue:
"accounts.profile" route inefficient for users with long project/release history
eats ram, times out, causes us infra trouble :)

we should probably trim down the query in https://github.com/pypa/warehouse/blob/81abc28f1d876ff2e80c5a4577eec4c3a5aee7b9/warehouse/accounts/views.py#L82-L89 to get the minimum amount of data necessary.


Discussion:
@dstufft suggested [this reference on orm.load_only](http://docs.sqlalchemy.org/en/latest/orm/loading_columns.html#sqlalchemy.orm.load_only) and a possible solution of:

```python
.options(Load(Release).load_only("summary", "whatever"))
```

Resolving patch:
diff --git a/warehouse/accounts/views.py b/warehouse/accounts/views.py
--- a/warehouse/accounts/views.py
+++ b/warehouse/accounts/views.py
@@ -19,7 +19,7 @@
 )
 from pyramid.security import Authenticated, remember, forget
 from pyramid.view import view_config
-from sqlalchemy.orm import joinedload
+from sqlalchemy.orm import Load
 from sqlalchemy.orm.exc import NoResultFound
 
 from warehouse.accounts import REDIRECT_FIELD_NAME
@@ -81,11 +81,11 @@ def profile(user, request):
 
     projects = (
         request.db.query(Project)
-                  .filter(Project.users.contains(user))
-                  .options(joinedload(Project.releases))
-                  .join(Release)
-                  .order_by(Release.created.desc())
-                  .all()
+        .filter(Project.users.contains(user))
+        .join(Project.releases)
+        .options(Load(Release).load_only('summary', 'created'))
+        .order_by(Release.created.desc())
+        .all()
     )
 
     return {"user": user, "projects": projects}


Test patch:
Repository: pypi/warehouse

Issue:
Change "Edit" to "Manage" in "Your Projects"
Change the button/link text "Edit" to "Manage" in "Your Projects".

From IRC conversation with @alanbato, @ewdurbin and @nlhkabu .

~~~
<EWDurbin> I think perhaps “Manage” might be a better name for the button that currently says “Edit"
<EWDurbin> Just right off the bat. Since well you can’t really Edit anything, just delete files/releases/projects
<di_codes> ^ agreed
<alanbato> Makes sense to me, Edit misguides people into thinking they can change project attributes imho
<nlh> yep 100% agree
<sumanah> nlh: and I agree with them but I want to hear your thoughts -- you're the one who's done user testing, so do you think people would understand "manage"?
<nlh> i'll open a PR :)
<nlh> yes
<sumanah> thanks nlh!
<nlh> it's also more consistent with the URL structure
~~~

But I do not see a pull request from Nicole yet, so I declare this a:

**Good First Issue**: This issue is good for first time contributors. If there is not a corresponding pull request for this issue, it is up for grabs. For directions for getting set up, see our [Getting Started Guide](https://warehouse.pypa.io/development/getting-started/). If you are working on this issue and have questions, please feel free to ask them here, in [`#pypa-dev` on Freenode](https://webchat.freenode.net/?channels=%23pypa-dev), or on the [pypa-dev mailing list](https://groups.google.com/forum/#!forum/pypa-dev).
Fix for #3118
Fixes #3118.

Updated the projects.html and the relating sass snippet to show **manage** in stead of **edit** when in **Your projects**.


Discussion:

![screenshot from 2018-03-02 21-57-52](https://user-images.githubusercontent.com/19406/36921478-d4654ec4-1e64-11e8-8b0a-48ae4a5dff9f.png)

Thanks @jw!

Could you also update this line here: https://github.com/pypa/warehouse/blob/master/warehouse/templates/includes/edit-project-button.html#L16.

(And it seems like that template could probably stand to be renamed as well.)

Resolving patch:
diff --git a/warehouse/packaging/views.py b/warehouse/packaging/views.py
--- a/warehouse/packaging/views.py
+++ b/warehouse/packaging/views.py
@@ -132,7 +132,7 @@ def release_detail(release, request):
 
 @view_config(
     route_name="includes.edit-project-button",
-    renderer="includes/edit-project-button.html",
+    renderer="includes/manage-project-button.html",
     uses_session=True,
     permission="manage",
 )


Test patch: