Repository: pyca/cryptography

Issue:
Implement __hash__ on all extensions we've implemented __eq__ on
We've fixed this haphazardly in the past, but we should probably consistently implement `__hash__` so extensions don't have odd behaviors like not being able to be inserted into hashable collections.
- [x] CRLNumber (#2580)
- [x] AuthorityKeyIdentifier
- [x] SubjectKeyIdentifier (#2583)
- [x] AuthorityInformationAccess
- [x] AccessDescription
- [x] BasicConstraints (#2594)
- [x] CRLDistributionPoints
- [x] DistributionPoint
- [x] CertificatePolicies
- [x] PolicyInformation
- [x] UserNotice
- [x] NoticeReference
- [x] ExtendedKeyUsage
- [x] InhibitAnyPolicy
- [x] KeyUsage
- [x] NameConstraints
- [x] Extension
- [x] PolicyConstraints
- [x] GeneralNames
- [x] SubjectAlternativeName
- [x] IssuerAlternativeName
- [x] CRLReason (#2595)
- [x] InvalidityDate
- [x] CertificateIssuer
- [x] PrecertificateSignedCertificateTimestamps
- [x] TLSFeature


Discussion:
Many of the remaining extensions use a list of items as their internal storage. Do we want to do something like hash(tuple(list))...? This could potentially be pretty expensive.

bleh. let's take some time and think for those, I don't think we have
external requests for these features yet, so we can take our time

On Sun, Dec 27, 2015 at 5:00 PM, Paul Kehrer notifications@github.com
wrote:

> Many of the remaining extensions use a list of items as their internal
> storage. Do we want to do something like hash(tuple(list))...? This could
> potentially be pretty expensive.
> 
> —
> Reply to this email directly or view it on GitHub
> https://github.com/pyca/cryptography/issues/2578#issuecomment-167444021.

## 

"I disapprove of what you say, but I will defend to the death your right to
say it." -- Evelyn Beatrice Hall (summarizing Voltaire)
"The people's good is the highest law." -- Cicero
GPG Key fingerprint: 125F 5C67 DFE9 4084

I'm going to preemptively apologize for an unintentionally long question, but: for those classes using lists for internal storage, is it a big deal if `__hash__` just reinstates `object.__hash__`? 

This would have the consequence that if, for example, two `GeneralNames` instances had the same internal `self._general_names` list, they would hash differently, and therefore be put into different buckets in dicts, etc. I'm asking because I honestly don't know: is it likely that this could result in problems?

I'm thinking about chipping away at a few of these (at what could only be described as a glacial pace, but hey, anything helps, and if I ended up blocking anyone I'd speed up), but I'm concerned about not violating the edict of "thou [shalt not](http://stackoverflow.com/questions/23161035/what-happens-if-an-objects-hash-changes) have [changing hashes](http://stackoverflow.com/questions/23161035/what-happens-if-an-objects-hash-changes)". If the internal storage iterables are ever mutated, it seems to me that the only safe option would be either to use `object.__hash__` directly, or something functionally equivalent (a unique hash for each instance regardless of internal state).

I suppose there are some other strategies that could be used, too, but they all seem fairly dangerous. At a certain point, if two equivalent objects really, really should be put in the same hash bucket, it seems to me like it'd be better off to convert the internal list (ex `GeneralNames._general_names`) into a tuple to enforce immutability. But that has the potential to break things, and I'm definitely not familiar enough with the project internals to actually suggest that.

Thoughts?

@alex I saw this issue as a starter project, and I would like to work on implementing `__hash__` for the remaining extensions. Would you prefer each extension as its own PR, or would you like to group them together as a PR addressing this particular issue (#2578)?
Individual PRs please, it makes them much easier to review and get merged quickly! Thanks for looking into contributing!
@alex I have been thinking about the comment from @Badg regarding `__hash__()` on classes with lists. I also consulted the [Python documentation](https://docs.python.org/2/reference/datamodel.html):

> If a class defines mutable objects and implements a `__cmp__()` or `__eq__()` method, it should not implement `__hash__()`, since hashable collection implementations require that an object’s hash value is immutable (if the object’s hash value changes, it will be in the wrong hash bucket).

My thoughts: since the data members of the AuthorityKeyIdentifier class (for example) are read-only, why don't we just construct the problematic members as tuples instead of lists?

In the AuthorityKeyIdentifier example; instead of:

``` python
if authority_cert_issuer is not None:
            authority_cert_issuer = list(authority_cert_issuer)
```

We could just do:

``` python
if authority_cert_issuer is not None:
            authority_cert_issuer = tuple(authority_cert_issuer)
```

This way, we wouldn't have to convert the list to a tuple every time `__hash__()` is called on these classes.
The attribute really shouldn't be a `tuple` -- `tuples` are not just immutable `lists`, they should not be treated as general purpose sequences. Until someone has a specific performance complaint, I'd much rather we convert to `tuple` in `__hash__`.
Completing this is now blocked on finishing #3461 

Resolving patch:
diff --git a/src/cryptography/hazmat/backends/openssl/x509.py b/src/cryptography/hazmat/backends/openssl/x509.py
--- a/src/cryptography/hazmat/backends/openssl/x509.py
+++ b/src/cryptography/hazmat/backends/openssl/x509.py
@@ -534,3 +534,23 @@ def entry_type(self):
         # we only have precerts.
         assert entry_type == self._backend._lib.CT_LOG_ENTRY_TYPE_PRECERT
         return x509.certificate_transparency.LogEntryType.PRE_CERTIFICATE
+
+    @property
+    def _signature(self):
+        ptrptr = self._backend._ffi.new("unsigned char **")
+        res = self._backend._lib.SCT_get0_signature(self._sct, ptrptr)
+        self._backend.openssl_assert(res > 0)
+        self._backend.openssl_assert(ptrptr[0] != self._backend._ffi.NULL)
+        return self._backend._ffi.buffer(ptrptr[0], res)[:]
+
+    def __hash__(self):
+        return hash(self._signature)
+
+    def __eq__(self, other):
+        if not isinstance(other, _SignedCertificateTimestamp):
+            return NotImplemented
+
+        return self._signature == other._signature
+
+    def __ne__(self, other):
+        return not self == other
diff --git a/src/cryptography/x509/extensions.py b/src/cryptography/x509/extensions.py
--- a/src/cryptography/x509/extensions.py
+++ b/src/cryptography/x509/extensions.py
@@ -1402,6 +1402,21 @@ def __repr__(self):
             )
         )
 
+    def __hash__(self):
+        return hash(tuple(self._signed_certificate_timestamps))
+
+    def __eq__(self, other):
+        if not isinstance(other, PrecertificateSignedCertificateTimestamps):
+            return NotImplemented
+
+        return (
+            self._signed_certificate_timestamps ==
+            other._signed_certificate_timestamps
+        )
+
+    def __ne__(self, other):
+        return not self == other
+
 
 @utils.register_interface(ExtensionType)
 class OCSPNonce(object):


Test patch:
diff --git a/tests/x509/test_x509_ext.py b/tests/x509/test_x509_ext.py
--- a/tests/x509/test_x509_ext.py
+++ b/tests/x509/test_x509_ext.py
@@ -4469,6 +4469,85 @@ def test_generate(self, backend):
         assert isinstance(poison, x509.PrecertPoison)
 
 
+@pytest.mark.requires_backend_interface(interface=RSABackend)
+@pytest.mark.requires_backend_interface(interface=X509Backend)
+class TestSignedCertificateTimestamps(object):
+    @pytest.mark.supported(
+        only_if=lambda backend: (
+            backend._lib.CRYPTOGRAPHY_OPENSSL_110F_OR_GREATER),
+        skip_message="Requires OpenSSL 1.1.0f+",
+    )
+    def test_eq(self, backend):
+        sct = _load_cert(
+            os.path.join("x509", "badssl-sct.pem"),
+            x509.load_pem_x509_certificate,
+            backend
+        ).extensions.get_extension_for_class(
+            x509.PrecertificateSignedCertificateTimestamps
+        ).value[0]
+        sct2 = _load_cert(
+            os.path.join("x509", "badssl-sct.pem"),
+            x509.load_pem_x509_certificate,
+            backend
+        ).extensions.get_extension_for_class(
+            x509.PrecertificateSignedCertificateTimestamps
+        ).value[0]
+        assert sct == sct2
+
+    @pytest.mark.supported(
+        only_if=lambda backend: (
+            backend._lib.CRYPTOGRAPHY_OPENSSL_110F_OR_GREATER),
+        skip_message="Requires OpenSSL 1.1.0f+",
+    )
+    def test_ne(self, backend):
+        sct = _load_cert(
+            os.path.join("x509", "badssl-sct.pem"),
+            x509.load_pem_x509_certificate,
+            backend
+        ).extensions.get_extension_for_class(
+            x509.PrecertificateSignedCertificateTimestamps
+        ).value[0]
+        sct2 = _load_cert(
+            os.path.join("x509", "cryptography-scts.pem"),
+            x509.load_pem_x509_certificate,
+            backend
+        ).extensions.get_extension_for_class(
+            x509.PrecertificateSignedCertificateTimestamps
+        ).value[0]
+        assert sct != sct2
+        assert sct != object()
+
+    @pytest.mark.supported(
+        only_if=lambda backend: (
+            backend._lib.CRYPTOGRAPHY_OPENSSL_110F_OR_GREATER),
+        skip_message="Requires OpenSSL 1.1.0f+",
+    )
+    def test_hash(self, backend):
+        sct = _load_cert(
+            os.path.join("x509", "badssl-sct.pem"),
+            x509.load_pem_x509_certificate,
+            backend
+        ).extensions.get_extension_for_class(
+            x509.PrecertificateSignedCertificateTimestamps
+        ).value[0]
+        sct2 = _load_cert(
+            os.path.join("x509", "badssl-sct.pem"),
+            x509.load_pem_x509_certificate,
+            backend
+        ).extensions.get_extension_for_class(
+            x509.PrecertificateSignedCertificateTimestamps
+        ).value[0]
+        sct3 = _load_cert(
+            os.path.join("x509", "cryptography-scts.pem"),
+            x509.load_pem_x509_certificate,
+            backend
+        ).extensions.get_extension_for_class(
+            x509.PrecertificateSignedCertificateTimestamps
+        ).value[0]
+        assert hash(sct) == hash(sct2)
+        assert hash(sct) != hash(sct3)
+
+
 @pytest.mark.requires_backend_interface(interface=RSABackend)
 @pytest.mark.requires_backend_interface(interface=X509Backend)
 class TestPrecertificateSignedCertificateTimestampsExtension(object):
@@ -4481,6 +4560,81 @@ def test_repr(self):
             "<PrecertificateSignedCertificateTimestamps([])>"
         )
 
+    @pytest.mark.supported(
+        only_if=lambda backend: (
+            backend._lib.CRYPTOGRAPHY_OPENSSL_110F_OR_GREATER),
+        skip_message="Requires OpenSSL 1.1.0f+",
+    )
+    def test_eq(self, backend):
+        psct1 = _load_cert(
+            os.path.join("x509", "badssl-sct.pem"),
+            x509.load_pem_x509_certificate,
+            backend
+        ).extensions.get_extension_for_class(
+            x509.PrecertificateSignedCertificateTimestamps
+        ).value
+        psct2 = _load_cert(
+            os.path.join("x509", "badssl-sct.pem"),
+            x509.load_pem_x509_certificate,
+            backend
+        ).extensions.get_extension_for_class(
+            x509.PrecertificateSignedCertificateTimestamps
+        ).value
+        assert psct1 == psct2
+
+    @pytest.mark.supported(
+        only_if=lambda backend: (
+            backend._lib.CRYPTOGRAPHY_OPENSSL_110F_OR_GREATER),
+        skip_message="Requires OpenSSL 1.1.0f+",
+    )
+    def test_ne(self, backend):
+        psct1 = _load_cert(
+            os.path.join("x509", "cryptography-scts.pem"),
+            x509.load_pem_x509_certificate,
+            backend
+        ).extensions.get_extension_for_class(
+            x509.PrecertificateSignedCertificateTimestamps
+        ).value
+        psct2 = _load_cert(
+            os.path.join("x509", "badssl-sct.pem"),
+            x509.load_pem_x509_certificate,
+            backend
+        ).extensions.get_extension_for_class(
+            x509.PrecertificateSignedCertificateTimestamps
+        ).value
+        assert psct1 != psct2
+        assert psct1 != object()
+
+    @pytest.mark.supported(
+        only_if=lambda backend: (
+            backend._lib.CRYPTOGRAPHY_OPENSSL_110F_OR_GREATER),
+        skip_message="Requires OpenSSL 1.1.0f+",
+    )
+    def test_hash(self, backend):
+        psct1 = _load_cert(
+            os.path.join("x509", "badssl-sct.pem"),
+            x509.load_pem_x509_certificate,
+            backend
+        ).extensions.get_extension_for_class(
+            x509.PrecertificateSignedCertificateTimestamps
+        ).value
+        psct2 = _load_cert(
+            os.path.join("x509", "badssl-sct.pem"),
+            x509.load_pem_x509_certificate,
+            backend
+        ).extensions.get_extension_for_class(
+            x509.PrecertificateSignedCertificateTimestamps
+        ).value
+        psct3 = _load_cert(
+            os.path.join("x509", "cryptography-scts.pem"),
+            x509.load_pem_x509_certificate,
+            backend
+        ).extensions.get_extension_for_class(
+            x509.PrecertificateSignedCertificateTimestamps
+        ).value
+        assert hash(psct1) == hash(psct2)
+        assert hash(psct1) != hash(psct3)
+
     @pytest.mark.supported(
         only_if=lambda backend: (
             backend._lib.CRYPTOGRAPHY_OPENSSL_110F_OR_GREATER),