Repository: pypi/warehouse

Issue:
Catch additional pymacaroons.Macaroon.deserialize exceptions
It appears that someone has [enumerated the various exceptions `pymacaroons.Macaroon.deserialize` might raise](https://github.com/ecordell/pymacaroons/issues/50). It'd be great if that were resolved, but we might want to further harden the work from #7424 to handle these other cases?

_Originally posted by @ewdurbin in https://github.com/pypa/warehouse/issues/7298#issuecomment-589957864_

---

**Good First Issue**: This issue is good for first time contributors. If you've already contributed to Warehouse, work on [another issue without this label](https://github.com/pypa/warehouse/issues?utf8=%E2%9C%93&q=is%3Aissue+is%3Aopen+-label%3A%22good+first+issue%22) instead. If there is not a corresponding pull request for this issue, it is up for grabs. For directions for getting set up, see our [Getting Started Guide](https://warehouse.pypa.io/development/getting-started/). If you are working on this issue and have questions, feel free to ask them here, [`#pypa-dev` on Freenode](https://webchat.freenode.net/?channels=%23pypa-dev), or the [pypa-dev mailing list](https://groups.google.com/forum/#!forum/pypa-dev).


Discussion:
I've added the ["good first issue"](https://github.com/pypa/warehouse/labels/good%20first%20issue) label to this issue.
In #7876 we discovered that `pymacaroons.Macaroon.deserialize` can also raise an `IndexError`, which is causing https://sentry.io/share/issue/8737571d268f41ae9eef2d12279882d2/
I want to work on this PR, but it would be great if I get some additional context before I start. 

With that in mind, I will outline below what I could understand about the problem until now. Please feel free to expand/add on if I have missed something here for an initial stab at this.

We are currently using `pymacaroons.Macaroon.deserialize` [here](https://github.com/pypa/warehouse/blob/master/warehouse/macaroons/services.py#L80) and [here](https://github.com/pypa/warehouse/blob/master/warehouse/macaroons/services.py#L105) , and we are not catching the other exceptions mentioned in https://github.com/ecordell/pymacaroons/issues/50.

So in order to work towards fixing this, we will had to add one `except` clause around both call sites per PR, and then 

1. return None 

https://github.com/pypa/warehouse/blob/2d6364a12372cbf3ff18aeb861f26432486bb88e/warehouse/macaroons/services.py#L79-L82

2. raise an exception again

https://github.com/pypa/warehouse/blob/2d6364a12372cbf3ff18aeb861f26432486bb88e/warehouse/macaroons/services.py#L104-L107

3. log an exception via LOGGER.error (The message in cases 2 and 3 will have to be discussed)

And finally based on how we handle it, add unit tests for it. We can also add a helpful FAQ to suggest the user on how he can further debug the issue, on a case-by-case basis.



Resolving patch:
diff --git a/warehouse/macaroons/services.py b/warehouse/macaroons/services.py
--- a/warehouse/macaroons/services.py
+++ b/warehouse/macaroons/services.py
@@ -13,6 +13,7 @@
 import binascii
 import datetime
 import json
+import struct
 import uuid
 
 import pymacaroons
@@ -67,20 +68,32 @@ def find_macaroon(self, macaroon_id):
 
         return dm
 
+    def _deserialize_raw_macaroon(self, raw_macaroon):
+        raw_macaroon = self._extract_raw_macaroon(raw_macaroon)
+
+        if raw_macaroon is None:
+            raise InvalidMacaroon("malformed or nonexistent macaroon")
+
+        try:
+            return pymacaroons.Macaroon.deserialize(raw_macaroon)
+        except (
+            IndexError,
+            TypeError,
+            ValueError,
+            binascii.Error,
+            struct.error,
+            MacaroonDeserializationException,
+        ):
+            raise InvalidMacaroon("malformed macaroon")
+
     def find_userid(self, raw_macaroon):
         """
         Returns the id of the user associated with the given raw (serialized)
         macaroon.
         """
-        raw_macaroon = self._extract_raw_macaroon(raw_macaroon)
-        if raw_macaroon is None:
-            return None
-
         try:
-            m = pymacaroons.Macaroon.deserialize(raw_macaroon)
-        except binascii.Error:
-            return None
-        except MacaroonDeserializationException:
+            m = self._deserialize_raw_macaroon(raw_macaroon)
+        except InvalidMacaroon:
             return None
 
         dm = self.find_macaroon(m.identifier.decode())
@@ -97,15 +110,7 @@ def verify(self, raw_macaroon, context, principals, permission):
 
         Raises InvalidMacaroon if the macaroon is not valid.
         """
-        raw_macaroon = self._extract_raw_macaroon(raw_macaroon)
-        if raw_macaroon is None:
-            raise InvalidMacaroon("malformed or nonexistent macaroon")
-
-        try:
-            m = pymacaroons.Macaroon.deserialize(raw_macaroon)
-        except MacaroonDeserializationException:
-            raise InvalidMacaroon("malformed macaroon")
-
+        m = self._deserialize_raw_macaroon(raw_macaroon)
         dm = self.find_macaroon(m.identifier.decode())
 
         if dm is None:


Test patch:
diff --git a/tests/unit/macaroons/test_services.py b/tests/unit/macaroons/test_services.py
--- a/tests/unit/macaroons/test_services.py
+++ b/tests/unit/macaroons/test_services.py
@@ -10,6 +10,9 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 
+import binascii
+import struct
+
 from unittest import mock
 from uuid import uuid4
 
@@ -17,6 +20,8 @@
 import pymacaroons
 import pytest
 
+from pymacaroons.exceptions import MacaroonDeserializationException
+
 from warehouse.macaroons import services
 from warehouse.macaroons.models import Macaroon
 
@@ -144,6 +149,40 @@ def test_verify_invalid_macaroon(self, monkeypatch, user_service, macaroon_servi
             pretend.call(mock.ANY, context, principals, permissions)
         ]
 
+    def test_deserialize_raw_macaroon_when_none(self, macaroon_service):
+        raw_macaroon = pretend.stub()
+        macaroon_service._extract_raw_macaroon = pretend.call_recorder(lambda a: None)
+
+        with pytest.raises(services.InvalidMacaroon):
+            macaroon_service._deserialize_raw_macaroon(raw_macaroon)
+
+        assert macaroon_service._extract_raw_macaroon.calls == [
+            pretend.call(raw_macaroon),
+        ]
+
+    @pytest.mark.parametrize(
+        "exception",
+        [
+            IndexError,
+            TypeError,
+            ValueError,
+            binascii.Error,
+            struct.error,
+            MacaroonDeserializationException,
+        ],
+    )
+    def test_deserialize_raw_macaroon(self, monkeypatch, macaroon_service, exception):
+        raw_macaroon = pretend.stub()
+        macaroon_service._extract_raw_macaroon = pretend.call_recorder(
+            lambda a: raw_macaroon
+        )
+        monkeypatch.setattr(
+            pymacaroons.Macaroon, "deserialize", pretend.raiser(exception)
+        )
+
+        with pytest.raises(services.InvalidMacaroon):
+            macaroon_service._deserialize_raw_macaroon(raw_macaroon)
+
     def test_verify_malformed_macaroon(self, macaroon_service):
         with pytest.raises(services.InvalidMacaroon):
             macaroon_service.verify(f"pypi-thiswillnotdeserialize", None, None, None)