Repository: pyca/cryptography

Issue:
decode_rfc6979_signature fails with some input
Here's a failing test:

``` diff
diff --git a/tests/hazmat/primitives/test_asym_utils.py b/tests/hazmat/primitives/test_asym_utils.py
index bf55bad..679464a 100644
--- a/tests/hazmat/primitives/test_asym_utils.py
+++ b/tests/hazmat/primitives/test_asym_utils.py
@@ -63,3 +63,10 @@ def test_decode_rfc6979_invalid_asn1():
         # This byte sequence has an invalid ASN.1 sequence length as well as
         # an invalid integer length for the second integer.
         decode_rfc6979_signature(b"0\x07\x02\x01\x01\x02\x02\x01")
+
+    with pytest.raises(ValueError):
+        decode_rfc6979_signature(
+            b" ,\x02\x14\x1b\x92\xd2_!\x81>[F\xfe\xf97\x08~\x9de\x85A\xa3\xeb"
+            b"\x02\x14\x08-\xd8\xb9\xbf\xb3M\xc6)rU\xc3+\xa6\xbc\x8c\x80V\x06"
+            b"\xe5"
+        )
```

I can't tell if this is our bug, or a pyasn.1 bug.

Discovered using afl-fuzz.



Discussion:
More minimal test case:

``` diff
diff --git a/tests/hazmat/primitives/test_asym_utils.py b/tests/hazmat/primitives/test_asym_utils.py
index bf55bad..c111708 100644
--- a/tests/hazmat/primitives/test_asym_utils.py
+++ b/tests/hazmat/primitives/test_asym_utils.py
@@ -63,3 +63,6 @@ def test_decode_rfc6979_invalid_asn1():
         # This byte sequence has an invalid ASN.1 sequence length as well as
         # an invalid integer length for the second integer.
         decode_rfc6979_signature(b"0\x07\x02\x01\x01\x02\x02\x01")
+
+    with pytest.raises(ValueError):
+        decode_rfc6979_signature(b"\x00\x81\x00")
```

Another bug, this one is definitely pyasn1:

```
'0,\x02\x14\x1b\x92\xd2_!\x81>[F\xfe\xf97\x08~\x9de\x85A\xa3\xeb\x02\x0f\x08-\xd8\xb9\xbf\xb3M\xc6)rU\xc3+\xa6\xe8\x03\x80\x02\x00\xe5'
```

is the input

Aaaand, another one:

```
'0,\x02\x14\x1b\x92\xd2_!\x81>[F\xfe\xf97\x08~\x9de\x85A\xa3\xeb\x02\x00#\x10\x00\x00\x00\xb3M\xc6)rU\xc3+\xa6\xbc\x8c\x80V\x06\xe5'
```

One more:

```
'0,\x02\x14\x1b\x92\xd2~\x9de\x85A\xa3\xeb\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x01\xfa\t\x01\xff\x02\x02\x02\x02\x02\x02\x02\x02\x02\x01\xbc\x8c\x80V\x06\xe5'
```

Another one:

```
'0,\x02\x14\x1b\x92\xd2_!\x81>[F\xfe\xf97\x08~\x9de\x85A\xa3\xeb\x02\x0f\x08-\xd8\xb9\xbf\xb3M\xc6)rU\xc3+\xa6\xe8\x03\x80\t\x00\xe5'
```

The first two are (arguably) a bug in cryptography's `decode_rfc6979_signature`: `decoder.decode` is capable of returning `EndOfOctets()` in the first element of the result tuple, instead of raising `PyAsn1Error` when it doesn't match the spec. I think you could just check for that.

The rest cause backtraces in standalone `decoder.decode()` from pyasn1 HEAD.

Is returning an `EndOfOctets`, when a spec is provided, a reasonable behavior? I guess I don't really understand what `EndOfOctets` means.

No. There are a few bugs here:
1. The end-of-contents octets are specified as being only `00 00`. The spec comments that they "can be considered as the encoding of" a hypothetical item with tag 0 (and primitive form), length 0, and no value. Another possible encoding of that value is `00 81 00`, encoding that same hypothetical thing with the long-form encoding of the length. But the wording of the spec doesn't let you interpret that as an end-of-contents.
2. The end-of-contents octets are only really sensible within the context of an indefinite-form, constructed-encoding object. (For instance, a sequence sent with indefinite length has its contents consist of a bunch of TLVs followed by `00 00`.) I suppose that it's somewhat sensible for the internal function to be able to return `endOfOctets` when recursively parsing a structure. I'm pretty sure it's not reasonable for the API function to do that _by default_.
3. Indefinite-form encodings are prohibited in DER anyway, so one would hope that something named `pyasn1.codec.der.decoder` wouldn't successfully parse the end-of-contents octets. For BER and CER it's fine.

But those are all bugs in pyasn1, and as far as cryptography is concerned, we should just check for `endOfOctets` and raise a `ValueError`. I can open a PR for that.

All of this is pretty firmly orthogonal to the logic errors that your later fuzzing found. Should we split this into two issues?

The last set are clearly bugs in pyasn1, so let's make a checklist of the
different errors that we want resolved upstream, and do a PR for the EOC
thing.

On Mon, Apr 13, 2015 at 4:59 PM, Geoffrey Thomas notifications@github.com
wrote:

> No. There are a few bugs here:
> 
>    1.
> 
>    The end-of-contents octets are specified as being only 00 00. The spec
>    comments that they "can be considered as the encoding of" a hypothetical
>    item with tag 0 (and primitive form), length 0, and no value. Another
>    possible encoding of that value is 00 81 00, encoding that same
>    hypothetical thing with the long-form encoding of the length. But the
>    wording of the spec doesn't let you interpret that as an end-of-contents.
>    2.
> 
>    The end-of-contents octets are only really sensible within the context
>    of an indefinite-form, constructed-encoding object. (For instance, a
>    sequence sent with indefinite length has its contents consist of a bunch of
>    TLVs followed by 00 00.) I suppose that it's somewhat sensible for the
>    internal function to be able to return endOfOctets when recursively
>    parsing a structure. I'm pretty sure it's not reasonable for the API
>    function to do that _by default_.
>    3.
> 
>    Indefinite-form encodings are prohibited in DER anyway, so one would
>    hope that something named pyasn1.codec.der.decoder wouldn't
>    successfully parse the end-of-contents octets. For BER and CER it's fine.
> 
> But those are all bugs in pyasn1, and as far as cryptography is concerned,
> we should just check for endOfOctets and raise a ValueError. I can open a
> PR for that.
> 
> All of this is pretty firmly orthogonal to the logic errors that your
> later fuzzing found. Should we split this into two issues?
> 
> —
> Reply to this email directly or view it on GitHub
> https://github.com/pyca/cryptography/issues/1838#issuecomment-92496719.

## 

"I disapprove of what you say, but I will defend to the death your right to
say it." -- Evelyn Beatrice Hall (summarizing Voltaire)
"The people's good is the highest law." -- Cicero
GPG Key fingerprint: 125F 5C67 DFE9 4084

```
'0,\x02\x14\x1b\x92\xd2_!\x81\x1c[F\xfe\xf97\x08~\x9de\x85A\xa3\xeb\x02\x01\xf9#\x08\t\x02~\x9de\xb9\xbfrU\xc3+\xa6\xbc\x8c\x80V\x06'
```

I think this is a distinct one.

The first example, `b" ,\x02\x14\x1b\x92\xd2_!\x81>[F\xfe\xf97\x08~\x9de\x85A\xa3\xeb\x02\x14\x08-\xd8\xb9\xbf\xb3M\xc6)rU\xc3+\xa6\xbc\x8c\x80V\x06\xe5"` is an object of type 0 (the first byte is `20`, i.e., constructed-form tag 0) that's just excessively long. Again, pyasn1 is parsing objects of type 0 as EOC, so the same fix should deal with that.

As a side note, pyasn1 doesn't implement an actual strict DER parser. It uses the same parser for CER and DER. This "CER"/"DER" parser is lightly modified from BER to add some of the restrictions common to both CER and DER (e.g., bools have to be `FF`, not just any nonzero value), but it otherwise successfully parses non-canonical / non-distinguished encodings. If that were fixed, your attack surface and fuzz surface would probably go down a lot.

> `'0,\x02\x14\x1b\x92\xd2_!\x81>[F\xfe\xf97\x08~\x9de\x85A\xa3\xeb\x02\x0f\x08-\xd8\xb9\xbf\xb3M\xc6)rU\xc3+\xa6\xe8\x03\x80\x02\x00\xe5'`

This is a sequence of three objects, of which the third one (`\x03\x80\x02\x00\xe5`) causes the crash. The proximate cause is that it's a constructed-encoding bit string (tag 3), which is supposed to recursively include other bit strings, but in this case it includes an integer (tag 2), which causes a Python-level type error. A clearer minimal example is `23 03 02 01 00`, a constructed-encoding bit string consisting of a single integer 0.

(There are also some non-proximate causes, like being willing to accept indefinite lengths when the tag byte claims primitive encoding, and hitting this code path at all for DER which forbids constructed encodings.)

> `'0,\x02\x14\x1b\x92\xd2_!\x81>[F\xfe\xf97\x08~\x9de\x85A\xa3\xeb\x02\x00#\x10\x00\x00\x00\xb3M\xc6)rU\xc3+\xa6\xbc\x8c\x80V\x06\xe5'`

Same sort of thing. The third object in this sequence (`#\x10\x00\x00...,` i.e., `23 10 00 00 ...`) is a constructed-encoding bit string that includes an EOC, so the `endOfOctets` sentinel is trying to get appended.

> `'0,\x02\x14\x1b\x92\xd2~\x9de\x85A\xa3\xeb\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x01\xfa\t\x01\xff\x02\x02\x02\x02\x02\x02\x02\x02\x02\x01\xbc\x8c\x80V\x06\xe5'`

The third object in this sequence is `09 01 ff` (`\t\x01\xff`), which is a malformed, invalid float. Although the real invalidity is that it's triggering a "reserved for further editions" case (X.690 clause 8.5.6.2), I think the proximate cause of the crash is that (nonzero) floats are expected to contain more than one byte, since the error is `IndexError: string index out of range`.

> `'0,\x02\x14\x1b\x92\xd2_!\x81>[F\xfe\xf97\x08~\x9de\x85A\xa3\xeb\x02\x0f\x08-\xd8\xb9\xbf\xb3M\xc6)rU\xc3+\xa6\xe8\x03\x80\t\x00\xe5'`

This one is like the ones in the previous comment: the third object (`\x03\x80\xt\x00\xe5`, i.e. `03 80 09 00 ...`) is an indefinite-length bitstring (incorrectly accepted) that starts with a float.

> `'0,\x02\x14\x1b\x92\xd2_!\x81\x1c[F\xfe\xf97\x08~\x9de\x85A\xa3\xeb\x02\x01\xf9#\x08\t\x02~\x9de\xb9\xbfrU\xc3+\xa6\xbc\x8c\x80V\x06'`

Also similar: `#\x08\t\x02~\x9de\xb9\xbfr` (`23 08 09 02 ...`) is a constructed-encoding bitstring whose first element is a float.

Another new looking failure:

```
'0\x80\x02\x14\x1b\x92\xb5_\x00\x00\x00\x10F\xfe\xf9V\x08~\x9de\x85A\xa3@\x02\x00\x04\x809\x80\t\x04\x950\x800\x00\x00\x00M'
```

All of these, as well as a directory of other crashing inputs that @alex found, are fixed in [my clone of pyasn1](/geofft/pyasn1). Want to look over those commits before I report these upstream?

I should probably make some test cases for use by both pyasn1 and cryptography.

@geofft The commits look good, please report upstream! Thanks so much!

:sparkles: :cake: :sparkles:


Resolving patch:
diff --git a/setup.py b/setup.py
--- a/setup.py
+++ b/setup.py
@@ -33,9 +33,9 @@
 
 requirements = [
     "idna>=2.0",
-    "pyasn1",
+    "pyasn1>=0.1.8",
     "six>=1.4.1",
-    "setuptools"
+    "setuptools",
 ]
 setup_requirements = []
 
diff --git a/src/cryptography/hazmat/primitives/asymmetric/utils.py b/src/cryptography/hazmat/primitives/asymmetric/utils.py
--- a/src/cryptography/hazmat/primitives/asymmetric/utils.py
+++ b/src/cryptography/hazmat/primitives/asymmetric/utils.py
@@ -4,7 +4,6 @@
 
 from __future__ import absolute_import, division, print_function
 
-from pyasn1.codec.ber import eoo
 from pyasn1.codec.der import decoder, encoder
 from pyasn1.error import PyAsn1Error
 from pyasn1.type import namedtype, univ
@@ -29,11 +28,6 @@ def decode_rfc6979_signature(signature):
         raise ValueError(
             "The signature contains bytes after the end of the ASN.1 sequence."
         )
-    # pyasn1 can erroneously return this from top-level DER decoding.
-    # It's intended as a sentinel in recursive BER decoding, so it's
-    # returned even though an asn1Spec is provided.
-    if eoo.endOfOctets.isSameTypeWith(data) and data == eoo.endOfOctets:
-        raise ValueError("Invalid signature data. Unable to decode ASN.1")
 
     r = int(data.getComponentByName('r'))
     s = int(data.getComponentByName('s'))


Test patch:
diff --git a/tests/hazmat/primitives/test_asym_utils.py b/tests/hazmat/primitives/test_asym_utils.py
--- a/tests/hazmat/primitives/test_asym_utils.py
+++ b/tests/hazmat/primitives/test_asym_utils.py
@@ -65,6 +65,6 @@ def test_decode_rfc6979_invalid_asn1():
         decode_rfc6979_signature(b"0\x07\x02\x01\x01\x02\x02\x01")
 
     with pytest.raises(ValueError):
-        # This is the BER "end-of-contents octets," which pyasn1 is
-        # wrongly willing to return from top-level DER decoding.
+        # This is the BER "end-of-contents octets," which older versions of
+        # pyasn1 are wrongly willing to return from top-level DER decoding.
         decode_rfc6979_signature(b"\x00\x00")