Repository: pypi/warehouse

Issue:
Incorrect http response from invalid upload endpoint
**Describe the bug**

As a user attempting to upload distributions to PyPI, I want to get a failure response from the PyPI http API when I attempt to upload files to an incorrect endpoint, so that I am able to correctly identify failures and successes and correct them if necessary or provide meaningful error messages.

The correct endpoint for uploading is `https://test.pypi.org/legacy/` (notice the trailing slash)

If an upload is attempted at `https://test.pypi.org/legacy` (without a trailing slash), the upload does not succeed, but a `200` response is returned.

**Expected behavior**

A 4xx or 5xx error code should be returned.

**To Reproduce**

Attempt to upload a distribution to `https://test.pypi.org/legacy`

**My Platform**

Mac, python 3.8, using `flit` as the upload front-end

**Additional context**



Discussion:
As far as I can tell this URL is correctly returning a 301 redirect in all cases:

```
$ curl -X POST https://test.pypi.org/legacy
<html>
 <head>
  <title>301 Moved Permanently</title>
 </head>
 <body>
  <h1>301 Moved Permanently</h1>
  The resource has been moved to /legacy/; you should be redirected automatically.


 </body>
</html>%
```

```
$ curl -I https://test.pypi.org/legacy
HTTP/2 301
content-security-policy: base-uri 'self'; block-all-mixed-content; connect-src 'self' https://api.github.com/repos/ *.fastly-insights.com sentry.io https://api.pwnedpasswords.com; default-src 'none'; font-src 'self' fonts.gstatic.com; form-action 'self'; frame-ancestors 'none'; frame-src 'none'; img-src 'self' https://warehouse-test-camo.ingress.cmh1.psfhosted.org/ www.google-analytics.com *.fastly-insights.com; script-src 'self' www.googletagmanager.com www.google-analytics.com *.fastly-insights.com https://cdn.ravenjs.com; style-src 'self' fonts.googleapis.com; worker-src *.fastly-insights.com
content-type: text/html; charset=UTF-8
location: https://test.pypi.org/legacy/
referrer-policy: origin-when-cross-origin
server: nginx/1.13.9
accept-ranges: bytes
date: Mon, 15 Jun 2020 15:47:07 GMT
age: 50
x-served-by: cache-bwi5128-BWI, cache-mci5929-MCI
x-cache: MISS, HIT
x-cache-hits: 0, 1
x-timer: S1592236028.844284,VS0,VE1
vary: Accept-Encoding
strict-transport-security: max-age=31536000; includeSubDomains; preload
x-frame-options: deny
x-xss-protection: 1; mode=block
x-content-type-options: nosniff
x-permitted-cross-domain-policies: none
x-robots-header: noindex
content-length: 205
```

Am I missing something?
POST requests typically don't return redirection responses because clients aren't supposed to follow them for non-GET/HEAD requests, or if they do follow them, they get changed to a GET request (which I suspect is what is happening here).

I'll do some debugging with the http client I'm using, which is `requests/urllib3`, to confirm this is the case.

It's one of those things where a 301 might technically be correct according to the http specification, but given the de facto behavior of all the http clients, it doesn't actually work the way it was intended.
That does indeed seem to be happening here:

```
>>> import requests
>>> resp = requests.post('https://test.pypi.org/legacy')
>>> resp.status_code
200
>>> resp.content
b'\n\n\n\n\n\n<!DOCTYPE html>\n<html lang="en" dir="ltr">\n  <head>\n    <meta charset="utf-8">\n    <meta http-equiv="X-UA-Compatible" content="IE=edge">\n    <meta name="viewport" content="width=device-width, initial-scale=1">\n\n    <meta name="defaultLanguage" content="en">\n    <meta name="availableLanguages" content="en, es, fr, ja, pt_BR, uk, el, de, zh_Hans, ru, he">\n\n    \n      \n    \n\n    <title>test.pypi.org \xc2\xb7 TestPyPI</title>\n    <meta name="description" content="The Python Package Index (PyPI) is a repository of software for the Python programming language.">\n\n    <link rel="stylesheet" href="/static/css/warehouse-ltr.f2d4f304.css">\n    <link rel="stylesheet" href="/static/css/fontawesome.6002a161.css">\n    <link rel="stylesheet" href="/static/css/regular.98fbf39a.css">\n    <link rel="stylesheet" href="/static/css/solid.c3b5f0b5.css">\n    <link rel="stylesheet" href="/static/css/brands.2c303be1.css">\n    <link rel="stylesheet" href="https://fonts.googleapis.com/css?family=Source+Sans+Pro:400,400italic,600,600italic,700,700italic%7CSource+Code+Pro:500">\n    <noscript>\n      <link rel="stylesheet" href="/static/css/noscript.14e57721.css">\n    </noscript>\n\n    \n\n    <link rel="icon" href="/static/images/favicon.6a76275d.ico" type="image/x-icon">\n\n    <link rel="alternate" type="application/rss+xml" title="RSS: 40 latest updates" href="/rss/updates.xml">\n    <link rel="alternate" type="application/rss+xml" title="RSS: 40 newest packages" href="/rss/packages.xml">\n    \n    \n\n    <meta property="og:url" content="https://test.pypi.org/legacy/">\n    <meta property="og:site_name" content="PyPI">\n    <meta property="og:type" content="website">\n    <meta property="og:image" content="https://test.pypi.org/static/images/twitter.90915068.jpg">\n    <meta property="og:title" content="test.pypi.org">\n    <meta property="og:description" content="The Python Package Index (PyPI) is a repository of software for the Python programming language.">\n\n    <link rel="search" type="application/opensearchdescription+xml" title="PyPI" href="/opensearch.xml">\n\n    \n    <script\n      src="https://cdn.ravenjs.com/3.26.2/raven.min.js"\n      integrity="sha384-D6LXy67EIC102DTuqypxwQsTHgiatlbvg7q/1YAWFb6lRyZ1lIZ6bGDsX7jxHNKA"\n      crossorigin="anonymous">\n    </script>\n    \n    <script async\n            data-sentry-frontend-dsn="https://d4c4740fc96e42b0908acb666d7ccde6@sentry.io/1231156"\n            src="/static/js/warehouse.943ec9d4.js">\n    </script>\n    \n    \n    <script defer src="https://www.fastly-insights.com/insights.js?k=6a52360a-f306-421e-8ed5-7417d0d4a4e9&dnt=true"></script>\n  </head>\n\n  <body data-controller="viewport-toggle">\n    \n<div class="viewport-section viewport-section--dark">\n  <div class="viewport-section__content">\n    <img alt="TestPyPI" src="/static/images/logo-small.6eef541e.svg">\n    <h2 class="viewport-section__heading">API endpoint</h2>\n    <div class="viewport-section__text no-top-margin">\n      <p>\n        This URL is an API endpoint for uploading files to PyPI.<br>\n        For more information on uploading projects to PyPI, visit the <a href="https://packaging.python.org/distributing/#uploading-your-project-to-pypi" title="External link" target="_blank" rel="noopener">Python Packaging User Guide</a>.\n      </p>\n      <p>Otherwise, we suggest you <a href="https://pypi.org/">go to the PyPI homepage</a>.</p>\n    </div>\n  </div>\n</div>\n\n  </body>\n\n</html>'
```

Would a 404 for `POST` requests be preferred? I.e., only issue the redirect for GET requests?
Seems like maybe whatever library is using `requests` could be setting `allow_redirects=False` instead: https://requests.readthedocs.io/en/latest/user/quickstart/#redirection-and-history
Yes, I think a 404 for `POST` requests would give more expected results. 

I'll update the `flit` issue to implement the `allow_redirects=False` so it can correctly handle either way.
https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/301 says:

> It is therefore recommended to use the `301` code only as a response for `GET` or `HEAD` methods and to use the `308 Permanent Redirect` for `POST` methods instead, as the method change is explicitly prohibited with this status.

So it seems like a 308 response would be the "most correct" thing for us to respond with here, although I'm not sure how `flit` would have handled that in this case.

I've added the "good first issue" label to this issue.
Hello all! I'll try and tackle this issue today at EuroPython Sprints! Hopefully I'll have a patch ready soon

Resolving patch:
diff --git a/warehouse/forklift/__init__.py b/warehouse/forklift/__init__.py
--- a/warehouse/forklift/__init__.py
+++ b/warehouse/forklift/__init__.py
@@ -40,6 +40,11 @@ def includeme(config):
     config.add_legacy_action_route(
         "forklift.legacy.doc_upload", "doc_upload", domain=forklift
     )
+
+    config.add_route(
+        "forklift.legacy.missing_trailing_slash", "/legacy", domain=forklift
+    )
+
     config.add_request_method(_help_url, name="help_url")
 
     if forklift:
diff --git a/warehouse/forklift/legacy.py b/warehouse/forklift/legacy.py
--- a/warehouse/forklift/legacy.py
+++ b/warehouse/forklift/legacy.py
@@ -32,7 +32,12 @@
 import wtforms
 import wtforms.validators
 
-from pyramid.httpexceptions import HTTPBadRequest, HTTPForbidden, HTTPGone
+from pyramid.httpexceptions import (
+    HTTPBadRequest,
+    HTTPForbidden,
+    HTTPGone,
+    HTTPPermanentRedirect,
+)
 from pyramid.response import Response
 from pyramid.view import view_config
 from sqlalchemy import exists, func, orm
@@ -180,10 +185,10 @@ def _valid_platform_tag(platform_tag):
 _valid_markdown_variants = {"CommonMark", "GFM"}
 
 
-def _exc_with_message(exc, message):
+def _exc_with_message(exc, message, **kwargs):
     # The crappy old API that PyPI offered uses the status to pass down
     # messages to the client. So this function will make that easier to do.
-    resp = exc(message)
+    resp = exc(detail=message, **kwargs)
     resp.status = "{} {}".format(resp.status_code, message)
     return resp
 
@@ -1504,3 +1509,21 @@ def doc_upload(request):
         "Uploading documentation is no longer supported, we recommend using "
         "https://readthedocs.org/.",
     )
+
+
+@view_config(
+    route_name="forklift.legacy.missing_trailing_slash",
+    require_csrf=False,
+    require_methods=["POST"],
+)
+def missing_trailing_slash_redirect(request):
+    """
+    Redirect requests to /legacy to the correct /legacy/ route with a
+    HTTP-308 Permanent Redirect
+    """
+    return _exc_with_message(
+        HTTPPermanentRedirect,
+        "An upload was attempted to /legacy but the expected upload URL is "
+        "/legacy/ (with a trailing slash)",
+        location=request.route_path("forklift.legacy.file_upload"),
+    )


Test patch:
diff --git a/tests/functional/legacy_api/test_basic.py b/tests/functional/legacy_api/test_basic.py
new file mode 100644
--- /dev/null
+++ b/tests/functional/legacy_api/test_basic.py
@@ -0,0 +1,29 @@
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+
+def test_incorrect_post_redirect(webtest):
+    """
+    Per issue #8104, we should issue an HTTP-308 for a POST
+    in /legacy and point the user to the correct endpoint,
+    /legacy/
+
+    See: https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/308
+    """
+    resp = webtest.post("/legacy", status=308)
+    assert resp.status == (
+        "308 An upload was attempted to /legacy but the expected upload URL is "
+        "/legacy/ (with a trailing slash)"
+    )
+
+    assert "location" in resp.headers
+    assert resp.headers["location"] == "http://localhost/legacy/"
diff --git a/tests/unit/forklift/test_init.py b/tests/unit/forklift/test_init.py
--- a/tests/unit/forklift/test_init.py
+++ b/tests/unit/forklift/test_init.py
@@ -31,6 +31,7 @@ def test_includeme(forklift_domain, monkeypatch):
         add_legacy_action_route=pretend.call_recorder(lambda *a, **k: None),
         add_template_view=pretend.call_recorder(lambda *a, **kw: None),
         add_request_method=pretend.call_recorder(lambda *a, **kw: None),
+        add_route=pretend.call_recorder(lambda *a, **kw: None),
     )
 
     forklift.includeme(config)
@@ -48,6 +49,13 @@ def test_includeme(forklift_domain, monkeypatch):
             "forklift.legacy.doc_upload", "doc_upload", domain=forklift_domain
         ),
     ]
+
+    assert config.add_route.calls == [
+        pretend.call(
+            "forklift.legacy.missing_trailing_slash", "/legacy", domain=forklift_domain
+        ),
+    ]
+
     assert config.add_request_method.calls == [pretend.call(_help_url, name="help_url")]
     if forklift_domain:
         assert config.add_template_view.calls == [
diff --git a/tests/unit/forklift/test_legacy.py b/tests/unit/forklift/test_legacy.py
--- a/tests/unit/forklift/test_legacy.py
+++ b/tests/unit/forklift/test_legacy.py
@@ -3430,3 +3430,17 @@ def test_doc_upload(pyramid_request):
         "410 Uploading documentation is no longer supported, we recommend "
         "using https://readthedocs.org/."
     )
+
+
+def test_missing_trailing_slash_redirect(pyramid_request):
+
+    pyramid_request.route_path = pretend.call_recorder(lambda *a, **kw: "/legacy/")
+
+    resp = legacy.missing_trailing_slash_redirect(pyramid_request)
+
+    assert resp.status_code == 308
+    assert resp.status == (
+        "308 An upload was attempted to /legacy but the expected upload URL is "
+        "/legacy/ (with a trailing slash)"
+    )
+    assert resp.headers["Location"] == "/legacy/"