Repository: pypi/warehouse

Issue:
Test pypi upload fails with 502 Bad Gateway error
**Describe the bug**
During a GitHub Action to release the latest version of aurora, the package fails to upload to the pypi test server with a 502 Bad Gateway error

**Expected behavior**
The Action has worked fine in the past

**To Reproduce**
https://github.com/fsciortino/Aurora/runs/4558019561?check_suite_focus=true

**My Platform**
GitHub Actions

Your help is greatly appreciated.  Mention @fsciortino .


Discussion:
Thanks for the report. This is fixed in #10497.
Your quick help is much appreciated. 🎉 
@di unfortunately it seems that there's still some issue: we still get the same error when trying to upload the `aurorafusion` package. 
https://github.com/fsciortino/Aurora/runs/4582247971?check_suite_focus=true

I tried to increase the package version number and create a new release via github, but the error is still `HTTPError: 502 Bad Gateway from https://test.pypi.org/legacy/`. Could you please have a look?
@fsciortino Unfortunately I'm not seeing the corresponding exception in our error tracker. Can you share the actual distribution file attempting to be uploaded with us here so I can debug?
@fsciortino Looks like this started failing more than 10 days ago, so it's not likely due to this particular issue: https://github.com/fsciortino/Aurora/runs/4468671514?check_suite_focus=true
@di thank you very much for looking into this. Yes, I've been seeing the same error for more than a week, and only 3 days ago @smithsp opened this issue because we couldn't see what could be going wrong on our side. We're trying to upload the package via
https://github.com/fsciortino/Aurora/blob/master/.github/workflows/python-publish.yml

How can I get a "distribution file" and share it with you?
I would recommend building your project locally (`python setup.py sdist`) and trying to upload it manually (`twine upload -r testpypi dist/aurorafusion-2.0.4.tar.gz`) to confirm that it also fails, then attach it here or host it elsewhere and link to it, whatever is easiest.
I confirm that `testpypi` also fails when doing a manual upload. Here's the dist file:

[aurorafusion-2.0.4.tar.gz](https://github.com/pypa/warehouse/files/7747164/aurorafusion-2.0.4.tar.gz)

I might be wrong, and it's certainly not a reason for crashing but it looks like you set a very long description, and no `long_description`. Did you mixup the 2 fields by any chance ?
I see this error locally:

```
$ twine upload --repository-url http://localhost/legacy/ aurorafusion-2.0.4.tar.gz
Uploading distributions to http://localhost/legacy/
Enter your username: di
Enter your password:
Uploading aurorafusion-2.0.4.tar.gz
100%|███████████████████████████████████████████| 11.5M/11.5M [00:05<00:00, 2.33MB/s]
NOTE: Try --verbose to see response content.
HTTPError: 400 Bad Request from http://localhost/legacy/
".. image:: https://badge.fury.io/py/aurorafusion.svg     :target: https://badge.fury.io/py/aurorafusion      .. image:: https://anaconda.org/conda-forge/aurorafusion/badges/version.svg        :target: https://anaconda.org/conda-forge/aurorafusion  .. image:: https://anaconda.org/conda-forge/aurorafusion/badges/latest_release_date.svg        :target: https://anaconda.org/conda-forge/aurorafusion  .. image:: https://anaconda.org/conda-forge/aurorafusion/badges/platforms.svg        :target: https://anaconda.org/conda-forge/aurorafusion  .. image:: https://anaconda.org/conda-forge/aurorafusion/badges/license.svg        :target: https://anaconda.org/conda-forge/aurorafusion  .. image:: https://anaconda.org/conda-forge/aurorafusion/badges/downloads.svg        :target: https://anaconda.org/conda-forge/aurorafusion  Aurora is a package to simulate heavy-ion transportm neutrals and radiation in magnetically-confined plasmas. It includes a 1.5D impurity transport forward model, thoroughly benchmarked with the widely-adopted STRAHL code. It also offers routines to analyze neutral states of hydrogen isotopes, both from the edge of fusion plasmas and from neutral beam injection. A simple interface to atomic data for fusion plasmas makes it a convenient tool for spectroscopy and integrated modeling. Aurora's code is mostly written in Python 3 and Fortran 90. An experimental Julia interface has also been added.   Documentation is available at https://aurora-fusion.readthedocs.io.   Development  -----------  The code is developed and maintained by F. Sciortino (MPI-IPP) in collaboration with T. Odstrcil (GA), D. Fajardo (MPI-IPP), A. Cavallaro (MIT) and R. Reksoatmodjo (W&M), with support from O. Linder (MPI-IPP), C. Johnson (U. Auburn), D. Stanczak (IPPLM) and S. Smith (GA). The STRAHL documentation provided by R.Dux (MPI-IPP) was extremely helpful to guide the initial development of Aurora.  New contributors are more than welcome! Please get in touch at francesco.sciortino-at-ipp.mpg.de or open a pull-request via Github.   Generally, we would appreciate if you could work with us to merge your features back into the main Aurora distribution if there is any chance that the changes that you made could be useful to others.   Installation ------------  Aurora can be installed from PyPI using      pip install aurorafusion --user      You can omit the `--user` flag if you have write-access to the default package directory on your system and wish to install there.  Installing via conda is also possible using      conda install -c conda-forge aurorafusion            Both the PyPI and conda installation are automatically updated at every package release. Note that the conda installation does not currently install dependencies on `omfit_classes`, which users may need to install via `pip` (see the `PyPI repo <https://pypi.org/project/omfit-classes/>`_).   To look at the code and contribute to the Aurora repository, it is recommended to install from source, by git-cloning the  `Aurora repo <https://github.com/fsciortino/aurora>`_ from Github. This will ensure that you can access the latest version of the tools.   For compilation after git-cloning, users can make use of the `setup.py` file, e.g. using       python setup.py -e .  or use the makefile in the package directory to build the Fortran code using       make clean; make     Note that the makefile will not install any of the dependencies, listed in the `requirements.txt` file in the main directory. You can use this file to quickly install dependencies within a Python virtual environment, or install each dependency one at a time.  The Julia version of the code is not built by default. If you have Julia installed on your system, you can do        make julia  from the main package directory. This will build a Julia `sysimage` to speed up access of Julia source code from Python, but it is not strictly necessary. See the documentation to read about interfacing Python and Julia.    Atomic data -----------  Aurora offers a simple interface to download, read, process and plot atomic data from the Atomic Data and Structure Analysis (ADAS) database, particularly through the OPEN-ADAS website: www.open-adas.ac.uk . ADAS data files can be fetched remotely and stored within the Aurora distribution directory, or users may choose to fetch ADAS files from a chosen, pre-existing directory by setting      export AURORA_ADAS_DIR=my_adas_directory      within their Linux environment (or analogous). If an ADAS files that is not available in AURORA_ADAS_DIR is requested by a user, Aurora attempts to download it and store it there. If you are using a public installation of Aurora and you do not have write-access to the directory where Aurora is installed, make sure to set AURORA_ADAS_DIR to a directory where you do have write-access before starting.  Several ADAS formats can currently be managed -- please see the docs. Please contact the authors to request and/or suggest expansions of current capabilities.    License -------  Aurora is distributed under the MIT License. The package is made open-source with the hope that this will speed up research on fusion energy and make further code development easier. However, we kindly ask that all users communicate to us their purposes, difficulties and successes with Aurora, so that we may support users as much as possible and grow the code further.    Citing Aurora -------------  Please see the `User Agreement <https://github.com/fsciortino/Aurora/blob/master/USER_AGREEMENT.txt>`_." is an invalid value for Summary. Error: Field cannot be longer than 512 characters. See https://packaging.python.org/specifications/core-metadata for more information.
```

Which seems like @ewjoachim is correct, but not sure why this results in a 502 from PyPI instead of a 400. Will reopen this to investigate (since this isn't due to #10497), but short-term changing `description` to `long_description` should fix this.
I think shortening/truncating the bad value in the error message here would probably help.
hmmm, from logs it looks like the backend _is_ returning a 400... 

```
[MY.IP.ADDR.ESS] - ewdurbin [20/Dec/2021:18:33:27 +0000] "POST /legacy/ HTTP/1.1" 400 17342 "-" "twine/3.4.2 importlib_metadata/4.8.1 pkginfo/1.7.1 requests/2.26.0 requests-toolbelt/0.9.1 tqdm/4.62.3 CPython/3.9.6"
```

I'll try to determine what's turning that into a 503...
Definitely looks like it's the fact that we're trying to shovel so much data into the HTTP Response Status line, which is a header. I'm guessing some part of our stack is choking on that, so @di's suggestion is a great start.

Resolving patch:
diff --git a/warehouse/forklift/legacy.py b/warehouse/forklift/legacy.py
--- a/warehouse/forklift/legacy.py
+++ b/warehouse/forklift/legacy.py
@@ -869,7 +869,12 @@ def file_upload(request):
             if field.description and isinstance(field, wtforms.StringField):
                 error_message = (
                     "{value!r} is an invalid value for {field}. ".format(
-                        value=field.data, field=field.description
+                        value=(
+                            field.data[:30] + "..." + field.data[-30:]
+                            if len(field.data) > 60
+                            else field.data
+                        ),
+                        field=field.description,
                     )
                     + "Error: {} ".format(form.errors[field_name][0])
                     + "See "


Test patch:
diff --git a/tests/unit/forklift/test_legacy.py b/tests/unit/forklift/test_legacy.py
--- a/tests/unit/forklift/test_legacy.py
+++ b/tests/unit/forklift/test_legacy.py
@@ -943,7 +943,11 @@ def test_fails_invalid_version(self, pyramid_config, pyramid_request, version):
                     "md5_digest": "a fake md5 digest",
                     "summary": "A" * 513,
                 },
-                "'" + "A" * 513 + "' is an invalid value for Summary. "
+                "'"
+                + "A" * 30
+                + "..."
+                + "A" * 30
+                + "' is an invalid value for Summary. "
                 "Error: Field cannot be longer than 512 characters. "
                 "See "
                 "https://packaging.python.org/specifications/core-metadata"