Repository: pypi/warehouse

Issue:
test.pypi.org returns 500 internal server error if user tries to upload malformed package (tar.gz)
Hi, thanks for your work. I found a bug when creating package. Here is how:
I was creating (actually generating) a python package. It got malformed (I need to learn to close files properly, always, :facepalm:). I tried to upload it to [test pypi](https://test.pypi.org) via [legacy api](https://warehouse.readthedocs.io/api-reference/legacy/#upload-api). Server responded with 500 internal server error.
response body: 
```
<html>
  <head>
    <title>Internal Server Error</title>
  </head>
  <body>
    <h1><p>Internal Server Error</p></h1>
    
  </body>
</html>
```

**Expected behavior**
400 with some helpful error message would be nice :slightly_smiling_face: 

**To Reproduce**
To reproduce this, one needs to call the API directly. Twine will fail before sending the request to server. I'm attaching the malformed package. btw, it really looks like ok archive:
```
$ file foo47-0.0.0.tar.gz
foo47-0.0.0.tar.gz: gzip compressed data
```

**Additional context**
malformed package: [foo47-0.0.0.tar.gz](https://github.com/pypa/warehouse/files/5028545/foo47-0.0.0.tar.gz)

---

**Good First Issue**: This issue is good for first time contributors. If you've already contributed to Warehouse, work on [another issue without this label](https://github.com/pypa/warehouse/issues?utf8=%E2%9C%93&q=is%3Aissue+is%3Aopen+-label%3A%22good+first+issue%22) instead. If there is not a corresponding pull request for this issue, it is up for grabs. For directions for getting set up, see our [Getting Started Guide](https://warehouse.pypa.io/development/getting-started/).


Discussion:
Indeed! Here's the exception it generated on our backend for completeness: https://sentry.io/share/issue/5c077b6c90b74b88b420028dc0493088/

(Marking this as "good first issue")
Is this still valid? The sentry link seems to be expired.
It's still valid, the sentry exceptions for TestPyPI seem to expire after 3 months.

I tried to upload the provided distribution but twine couldn't even handle the file (see https://github.com/pypa/twine/issues/712). @matusf, what version of Python / Twine did you use originally?
I think I found the error, anyone recall if it was an EOFError? I was able to trigger it via unit tests. I have a patch ready if so. 

via Hiptop

> On Nov 11, 2020, at 1:21 PM, Dustin Ingram <notifications@github.com> wrote:
> 
> ﻿
> It's still valid, the sentry exceptions for TestPyPI seem to expire after 3 months.
> 
> I tried to upload the provided distribution but twine couldn't even handle the file (see pypa/twine#712). @matusf, what version of Python / Twine did you use originally?
> 
> —
> You are receiving this because you commented.
> Reply to this email directly, view it on GitHub, or unsubscribe.

I haven't used twine. I've made the request directly to the legacy endpoint in Go. I may try to crash it one more time to show the exception in sentry :smiley:  
@matusf Please do!

Resolving patch:
diff --git a/warehouse/forklift/legacy.py b/warehouse/forklift/legacy.py
--- a/warehouse/forklift/legacy.py
+++ b/warehouse/forklift/legacy.py
@@ -647,7 +647,7 @@ def _is_valid_dist_file(filename, filetype):
                     member = tar.next()
                 if bad_tar:
                     return False
-        except tarfile.ReadError:
+        except (tarfile.ReadError, EOFError):
             return False
     elif filename.endswith(".exe"):
         # The only valid filetype for a .exe file is "bdist_wininst".


Test patch:
diff --git a/tests/unit/forklift/test_legacy.py b/tests/unit/forklift/test_legacy.py
--- a/tests/unit/forklift/test_legacy.py
+++ b/tests/unit/forklift/test_legacy.py
@@ -1853,6 +1853,50 @@ def test_upload_fails_with_invalid_file(self, pyramid_config, db_request):
         assert resp.status_code == 400
         assert resp.status == "400 Invalid distribution file."
 
+    def test_upload_fails_end_of_file_error(self, pyramid_config, db_request, metrics):
+        pyramid_config.testing_securitypolicy(userid=1)
+
+        user = UserFactory.create()
+        EmailFactory.create(user=user)
+        project = ProjectFactory.create(name="Package-Name")
+        RoleFactory.create(user=user, project=project)
+
+        # Malformed tar.gz, triggers EOF error
+        file_contents = b"\x8b\x08\x00\x00\x00\x00\x00\x00\xff"
+
+        db_request.user = user
+        db_request.user_agent = "warehouse-tests/6.6.6"
+        db_request.POST = MultiDict(
+            {
+                "metadata_version": "1.1",
+                "name": "malformed",
+                "version": "1.1",
+                "summary": "This is my summary!",
+                "filetype": "sdist",
+                "md5_digest": hashlib.md5(file_contents).hexdigest(),
+                "content": pretend.stub(
+                    filename="malformed-1.1.tar.gz",
+                    file=io.BytesIO(file_contents),
+                    type="application/tar",
+                ),
+            }
+        )
+
+        storage_service = pretend.stub(store=lambda path, filepath, meta: None)
+        db_request.find_service = lambda svc, name=None, context=None: {
+            IFileStorage: storage_service,
+            IMetricsService: metrics,
+        }.get(svc)
+        db_request.user_agent = "warehouse-tests/6.6.6"
+
+        with pytest.raises(HTTPBadRequest) as excinfo:
+            legacy.file_upload(db_request)
+
+        resp = excinfo.value
+
+        assert resp.status_code == 400
+        assert resp.status == "400 Invalid distribution file."
+
     def test_upload_fails_with_too_large_file(self, pyramid_config, db_request):
         pyramid_config.testing_securitypolicy(userid=1)