Repository: pypi/warehouse

Issue:
500 error when uploading with breached password
**Describe the bug**
Uploading package with twine fails with 500 Internal Server Error.

```
e:\Git\ui-map-parser>twine upload dist\ui_map_parser-1.0.0-py3-none-any.whl --verbose
Uploading distributions to https://upload.pypi.org/legacy/
  dist\ui_map_parser-1.0.0-py3-none-any.whl (10.0 KB)
Uploading ui_map_parser-1.0.0-py3-none-any.whl
100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 15.5k/15.5k [00:06<00:00, 2.54kB/s]
Received "500: Internal Server Error" Package upload appears to have failed.  Retry 1 of 5
Uploading ui_map_parser-1.0.0-py3-none-any.whl
100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 15.5k/15.5k [00:05<00:00, 2.93kB/s]
Received "500: Internal Server Error" Package upload appears to have failed.  Retry 2 of 5
Uploading ui_map_parser-1.0.0-py3-none-any.whl
100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 15.5k/15.5k [00:05<00:00, 2.73kB/s]
Received "500: Internal Server Error" Package upload appears to have failed.  Retry 3 of 5
Uploading ui_map_parser-1.0.0-py3-none-any.whl
100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 15.5k/15.5k [00:05<00:00, 2.93kB/s]
Received "500: Internal Server Error" Package upload appears to have failed.  Retry 4 of 5
Uploading ui_map_parser-1.0.0-py3-none-any.whl
100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 15.5k/15.5k [00:05<00:00, 2.81kB/s]
Received "500: Internal Server Error" Package upload appears to have failed.  Retry 5 of 5
Content received from server:
<html>
  <head>
    <title>Internal Server Error</title>
  </head>
  <body>
    <h1><p>Internal Server Error</p></h1>

  </body>
</html>

HTTPError: 500 Internal Server Error from https://upload.pypi.org/legacy/
Internal Server Error
```

`twine check dist\*` output shows that all is ok:
```
e:\Git\ui-map-parser>twine check dist\*
Checking dist\ui_map_parser-1.0.0-py3-none-any.whl: PASSED
Checking dist\ui-map-parser-1.0.0.tar.gz: PASSED
```

The same if trying to upload `ui-map-parser-1.0.0.tar.gz` file.

**Expected behavior**
Package should be uploaded succesfully.

**To Reproduce**
Execute `twine upload ui-map-parser-1.0.0.tar.gz`

**My Platform**
Windows 10.0.17763
Python 3.7.3
Twine 3.2.0

**Additional context**
Project repo: https://github.com/F1ashhimself/ui-map-parser
[ui-map-parser-1.0.0.tar.gz](https://github.com/pypa/warehouse/files/5074613/ui-map-parser-1.0.0.tar.gz)


Discussion:
I can't reproduce your issue with the `ui-map-parser-1.0.0.tar.gz` file provided. Can you provide the wheel as well?

Would it be possible for you to try this again and note the time so I can correlate this with the logs?
I cant add `.whl` file because this format is not supported, I just added `.zip` to end of file. You can remove this extension and operate like with `.whl` file.

[ui_map_parser-1.0.0-py3-none-any.whl.zip](https://github.com/pypa/warehouse/files/5076401/ui_map_parser-1.0.0-py3-none-any.whl.zip)

I can't reproduce with that file either, I'll need you to try again & let me know the time that you got the error.
Here are gif with all steps including creation of wheel and tar.gz files.
https://i.imgur.com/yAaRrZe.gif
@di also I have reproduced it one more time for you at 09:10 UTC 17 Aug 2020
I can correlate this with our logs, but all I see is this (with no stack trace):

```
pypi-warehouse-web-uploads-cdb98bcb-gqgg5 web-uploads  - - [17/Aug/2020:09:10:24 +0000] "POST /legacy/ HTTP/1.1" 500 0 "-" "-"
pypi-warehouse-web-uploads-cdb98bcb-s49qv web-uploads  - - [17/Aug/2020:09:10:18 +0000] "POST /legacy/ HTTP/1.1" 500 0 "-" "-"
```

And there's no corresponding issue in our alerting system, which is odd.
@di how I can help to figure out the problem?
@F1ashhimself Could you try to reproduce by uploading to https://test.pypi.org/?
@di I have deleted .pypirc file and created it again with token instead of password and uploaded this package to main pypi successfully. Thank you for your help.
@F1ashhimself That's interesting. Did your password contain any non-ascii characters or anything else unusual?
@di I have password with only characters and number but it was set long time ago, so when Im trying to login on site with this password I have received message that password does not meet requirements and should be changed. After changing it I tried only with token. Maybe the problem was in that old password that was need to be renewed.
@F1ashhimself Was the message that your password was included in a breach? PyPI integrates with https://haveibeenpwned.com/ to prevent using accounts with breached passwords. It sounds like this might not be working 100% correctly.
@di The message was something like "You password is insecure, you should add one or two words to it...". Nothing about breach but that password was compromised but not on my account.
@F1ashhimself Can you run the following script, changing the `password` variable to be the old password you were using, and let me know what it prints?

This will use https://haveibeenpwned.com/API/v2#PwnedPasswords to determine if PyPI would have considered your password breached. Your password will be hashed before it's sent to the API.

```python
import hashlib
import requests

password = "your password here"

hash_object = hashlib.sha1(password.encode("utf-8"))
digest = hash_object.hexdigest()
result = requests.get("https://api.pwnedpasswords.com/range/" + digest[:5]).text
print(digest[5:].upper() in result)
```
@di script returns `True` and as I said previously this password was leaked but message when logging in was not about leaked password.
Thanks for confirming! I've updated the issue title accordingly.

Resolving patch:
diff --git a/warehouse/email/__init__.py b/warehouse/email/__init__.py
--- a/warehouse/email/__init__.py
+++ b/warehouse/email/__init__.py
@@ -31,6 +31,20 @@ def _compute_recipient(user, email):
     return str(Address(first([user.name, user.username], default=""), addr_spec=email))
 
 
+def _redact_ip(request, email):
+    # We should only store/display IP address of an 'email sent' event if the user
+    # who triggered the email event is the one who receives the email. Else display
+    # 'Redacted' to prevent user privacy concerns. If we don't know the user who
+    # triggered the action, default to showing the IP of the source.
+    user_email = request.db.query(Email).filter(Email.email == email).one()
+
+    if request.unauthenticated_userid:
+        return user_email.user_id != request.unauthenticated_userid
+    if request.user:
+        return user_email.user_id != request.user.id
+    return False
+
+
 @tasks.task(bind=True, ignore_result=True, acks_late=True)
 def send_email(task, request, recipient, msg, success_event):
     msg = EmailMessage(**msg)
@@ -58,13 +72,6 @@ def _send_email_to_user(request, user, msg, *, email=None, allow_unverified=Fals
     if email is None or not (email.verified or allow_unverified):
         return
 
-    # We should only store/display IP address of an 'email sent' event if the user
-    # who triggered the email event is the one who receives the email. Else display
-    # 'Redacted' to prevent user privacy concerns. If we don't know the user who
-    # triggered the action, default to showing the IP of the source.
-    user_email = request.db.query(Email).filter(Email.email == email.email).one()
-    redact_ip = user_email.user_id != request.user.id if request.user else False
-
     request.task(send_email).delay(
         _compute_recipient(user, email.email),
         {
@@ -80,7 +87,7 @@ def _send_email_to_user(request, user, msg, *, email=None, allow_unverified=Fals
                 "from_": request.registry.settings.get("mail.sender"),
                 "to": email.email,
                 "subject": msg.subject,
-                "redact_ip": redact_ip,
+                "redact_ip": _redact_ip(request, email.email),
             },
         },
     )


Test patch:
diff --git a/tests/unit/email/test_init.py b/tests/unit/email/test_init.py
--- a/tests/unit/email/test_init.py
+++ b/tests/unit/email/test_init.py
@@ -59,6 +59,31 @@ def test_compute_recipient(user, address, expected):
     assert email._compute_recipient(user, email_) == expected
 
 
+@pytest.mark.parametrize(
+    ("unauthenticated_userid", "user", "expected"),
+    [
+        ("the_users_id", None, False),
+        ("some_other_id", None, True),
+        (None, pretend.stub(id="the_users_id"), False),
+        (None, pretend.stub(id="some_other_id"), True),
+        (None, None, False),
+    ],
+)
+def test_redact_ip(unauthenticated_userid, user, expected):
+    user_email = pretend.stub(user_id="the_users_id")
+
+    request = pretend.stub(
+        unauthenticated_userid=unauthenticated_userid,
+        user=user,
+        db=pretend.stub(
+            query=lambda a: pretend.stub(
+                filter=lambda a: pretend.stub(one=lambda: user_email)
+            )
+        ),
+    )
+    assert email._redact_ip(request, user_email) == expected
+
+
 class TestSendEmailToUser:
     @pytest.mark.parametrize(
         ("name", "username", "primary_email", "address", "expected"),
@@ -81,7 +106,7 @@ class TestSendEmailToUser:
         ],
     )
     def test_sends_to_user_with_verified(
-        self, name, username, primary_email, address, expected
+        self, name, username, primary_email, address, expected, pyramid_request
     ):
         user = pretend.stub(
             name=name,
@@ -91,28 +116,25 @@ def test_sends_to_user_with_verified(
         )
 
         task = pretend.stub(delay=pretend.call_recorder(lambda *a, **kw: None))
-        request = pretend.stub(
-            task=pretend.call_recorder(lambda x: task),
-            db=pretend.stub(
-                query=lambda a: pretend.stub(
-                    filter=lambda *a: pretend.stub(
-                        one=lambda: pretend.stub(user_id=user.id)
-                    )
-                ),
+        pyramid_request.task = pretend.call_recorder(lambda x: task)
+        pyramid_request.db = pretend.stub(
+            query=lambda a: pretend.stub(
+                filter=lambda *a: pretend.stub(
+                    one=lambda: pretend.stub(user_id=user.id)
+                )
             ),
-            remote_addr="0.0.0.0",
-            user=user,
-            registry=pretend.stub(settings={"mail.sender": "noreply@example.com"}),
         )
+        pyramid_request.user = user
+        pyramid_request.registry.settings = {"mail.sender": "noreply@example.com"}
 
         if address is not None:
             address = pretend.stub(email=address, verified=True)
 
         msg = EmailMessage(subject="My Subject", body_text="My Body")
 
-        email._send_email_to_user(request, user, msg, email=address)
+        email._send_email_to_user(pyramid_request, user, msg, email=address)
 
-        assert request.task.calls == [pretend.call(email.send_email)]
+        assert pyramid_request.task.calls == [pretend.call(email.send_email)]
         assert task.delay.calls == [
             pretend.call(
                 expected,
@@ -120,7 +142,7 @@ def test_sends_to_user_with_verified(
                 {
                     "tag": "account:email:sent",
                     "user_id": user.id,
-                    "ip_address": request.remote_addr,
+                    "ip_address": pyramid_request.remote_addr,
                     "additional": {
                         "from_": "noreply@example.com",
                         "to": address.email if address else primary_email,
@@ -171,7 +193,7 @@ def test_doesnt_send_with_unverified(self, primary_email, address):
         ],
     )
     def test_sends_unverified_with_override(
-        self, username, primary_email, address, expected
+        self, username, primary_email, address, expected, pyramid_request
     ):
         user = pretend.stub(
             username=username,
@@ -183,19 +205,16 @@ def test_sends_unverified_with_override(
         )
 
         task = pretend.stub(delay=pretend.call_recorder(lambda *a, **kw: None))
-        request = pretend.stub(
-            task=pretend.call_recorder(lambda x: task),
-            db=pretend.stub(
-                query=lambda a: pretend.stub(
-                    filter=lambda *a: pretend.stub(
-                        one=lambda: pretend.stub(user_id=user.id)
-                    )
-                ),
+        pyramid_request.task = pretend.call_recorder(lambda x: task)
+        pyramid_request.db = pretend.stub(
+            query=lambda a: pretend.stub(
+                filter=lambda *a: pretend.stub(
+                    one=lambda: pretend.stub(user_id=user.id)
+                )
             ),
-            remote_addr="0.0.0.0",
-            user=user,
-            registry=pretend.stub(settings={"mail.sender": "noreply@example.com"}),
         )
+        pyramid_request.user = user
+        pyramid_request.registry.settings = {"mail.sender": "noreply@example.com"}
 
         if address is not None:
             address = pretend.stub(email=address, verified=False)
@@ -203,10 +222,10 @@ def test_sends_unverified_with_override(
         msg = EmailMessage(subject="My Subject", body_text="My Body")
 
         email._send_email_to_user(
-            request, user, msg, email=address, allow_unverified=True
+            pyramid_request, user, msg, email=address, allow_unverified=True
         )
 
-        assert request.task.calls == [pretend.call(email.send_email)]
+        assert pyramid_request.task.calls == [pretend.call(email.send_email)]
         assert task.delay.calls == [
             pretend.call(
                 expected,
@@ -214,7 +233,7 @@ def test_sends_unverified_with_override(
                 {
                     "tag": "account:email:sent",
                     "user_id": user.id,
-                    "ip_address": request.remote_addr,
+                    "ip_address": pyramid_request.remote_addr,
                     "additional": {
                         "from_": "noreply@example.com",
                         "to": address.email if address else primary_email,