Repository: pyca/cryptography

Issue:
Cryptography 1.2.2 on OS X causing TypeError with AES-GCM cipher
Found this in unit tests we've been working on for our system. Reduced the problem down to the following example code, which causes a TypeError, sometimes 1 in 3 runs cause the failure, sometimes fails every time, you may have to run a few times to hit the problem.

``` python
from cryptography.hazmat.backends import default_backend
from cryptography.hazmat.primitives.ciphers import Cipher, algorithms, modes

cek = [0] * 32
iv = [0] * 12
tag = [0] * 16 

cipher = Cipher(algorithms.AES(cek), modes.GCM(iv, tag), backend=default_backend())
decryptor = cipher.decryptor()
```

as follows:

``` python
Traceback (most recent call last):
  File "fail.py", line 13, in <module>
    decryptor = cipher.decryptor()
  File "/Users/collis/.virtualenvs/jwt/lib/python3.5/site-packages/cryptography/hazmat/primitives/ciphers/base.py", line 115, in decryptor
    self.algorithm, self.mode
  File "/Users/collis/.virtualenvs/jwt/lib/python3.5/site-packages/cryptography/hazmat/backends/multibackend.py", line 59, in create_symmetric_decryption_ctx
    return b.create_symmetric_decryption_ctx(cipher, mode)
  File "/Users/collis/.virtualenvs/jwt/lib/python3.5/site-packages/cryptography/hazmat/backends/openssl/backend.py", line 875, in create_symmetric_decryption_ctx
    return _CipherContext(self, cipher, mode, _CipherContext._DECRYPT)
  File "/Users/collis/.virtualenvs/jwt/lib/python3.5/site-packages/cryptography/hazmat/backends/openssl/ciphers.py", line 86, in __init__
    len(mode.tag), mode.tag
TypeError: initializer for ctype 'void *' must be a cdata pointer, not list
```

I printed the values being passed to "self._backend._lib.EVP_CIPHER_CTX_ctrl()" above on error it gives:

``` python
<cdata 'EVP_CIPHER_CTX *' 0x7fa4fd83d980>, 17, 16, [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]
```

Note the [0, 0, 0...] is just the tag, changing the value changes this. I've tried this with actual values from our test case which found the problem e.g.

``` python
cek = [177, 161, 244, 128, 84, 143, 225, 115, 63, 180, 3, 255, 107, 154, 212, 246, 138, 7, 110, 91, 112, 46, 34, 105, 47, 130, 203, 46, 122, 234, 64, 252]
iv  = [227, 197, 117, 252, 2, 219, 233, 68, 180, 225, 77, 219]
tag = [92, 80, 104, 49, 133, 25, 161, 215, 173, 101, 219, 211, 136, 91, 210, 145]
```

Same result...
- OSX 10.11.2
- Python 3.5.1
- Cryptography 1.2.2

Tried Cryptography 1.1, 1.1.1, 1.1.2, 1.2, 1.2.1, 1.2.2, same problem.

Sometimes get the same error in a different location:

``` python
Traceback (most recent call last):
  File "fail.py", line 13, in <module>
    decryptor = cipher.decryptor()
  File "/Users/collis/.virtualenvs/jwt/lib/python3.5/site-packages/cryptography/hazmat/primitives/ciphers/base.py", line 115, in decryptor
    self.algorithm, self.mode
  File "/Users/collis/.virtualenvs/jwt/lib/python3.5/site-packages/cryptography/hazmat/backends/multibackend.py", line 59, in create_symmetric_decryption_ctx
    return b.create_symmetric_decryption_ctx(cipher, mode)
  File "/Users/collis/.virtualenvs/jwt/lib/python3.5/site-packages/cryptography/hazmat/backends/commoncrypto/backend.py", line 120, in create_symmetric_decryption_ctx
    self, cipher, mode, self._lib.kCCDecrypt
  File "/Users/collis/.virtualenvs/jwt/lib/python3.5/site-packages/cryptography/hazmat/backends/commoncrypto/ciphers.py", line 137, in __init__
    self._backend._ffi.NULL, 0, 0, 0, self._ctx)
TypeError: initializer for ctype 'void *' must be a cdata pointer, not list
```

Looks like something going on in OpenSSL/cffi?



Discussion:
Sorry to answer this about 15 min too late  - we weren't casting our list of integers to bytes correctly

```
from cryptography.hazmat.backends import default_backend
from cryptography.hazmat.primitives.ciphers import Cipher, algorithms, modes

cek = [177, 161, 244, 128, 84, 143, 225, 115, 63, 180, 3, 255, 107, 154, 212, 246, 138, 7, 110, 91, 112, 46, 34, 105, 47, 130, 203, 46, 122, 234, 64, 252]
iv  = [227, 197, 117, 252, 2, 219, 233, 68, 180, 225, 77, 219]
tag = [92, 80, 104, 49, 133, 25, 161, 215, 173, 101, 219, 211, 136, 91, 210, 145]



dave = Cipher(algorithms.AES(bytes(cek)), modes.GCM(bytes(iv), bytes(tag)), backend=default_backend()).decryptor()
```

I think what threw us for most of the day looking at this was the error message making us look at the backend and how we interacted with openssl rather than the simple issue we weren't giving the cipher algorithm bytes. 

Please close - not sure how you'd throw a more helpful error here, but willing to be told otherwise.

Wonder if this is something can be caught rather than failing in this way as it doesn't fail consistently?

Ok, plot thickens, if we only wrap the tag in bytes() but not the other values, this is when it fails randomly:

``` python
from cryptography.hazmat.backends import default_backend
from cryptography.hazmat.primitives.ciphers import Cipher, algorithms, modes

cek = [0] * 32
iv = [0] * 12
tag = [0] * 16

cipher = Cipher(algorithms.AES(cek), modes.GCM(iv, bytes(tag)), backend=default_backend())
decryptor = cipher.decryptor()
```

Have to run it a few times to cause the TypeError, would have thought this would fail every time given the nature of the error, a bit worrying?

``` python
Traceback (most recent call last):
  File "fail.py", line 13, in <module>
    decryptor = cipher.decryptor()
  File "/Users/collis/.virtualenvs/jwt/lib/python3.5/site-packages/cryptography/hazmat/primitives/ciphers/base.py", line 115, in decryptor
    self.algorithm, self.mode
  File "/Users/collis/.virtualenvs/jwt/lib/python3.5/site-packages/cryptography/hazmat/backends/multibackend.py", line 59, in create_symmetric_decryption_ctx
    return b.create_symmetric_decryption_ctx(cipher, mode)
  File "/Users/collis/.virtualenvs/jwt/lib/python3.5/site-packages/cryptography/hazmat/backends/commoncrypto/backend.py", line 120, in create_symmetric_decryption_ctx
    self, cipher, mode, self._lib.kCCDecrypt
  File "/Users/collis/.virtualenvs/jwt/lib/python3.5/site-packages/cryptography/hazmat/backends/commoncrypto/ciphers.py", line 137, in __init__
    self._backend._ffi.NULL, 0, 0, 0, self._ctx)
TypeError: initializer for ctype 'void *' must be a cdata pointer, not list
```

I can reproduce this. The reason it is intermittent on Py 3.5 is that when `default_backend` is called and the set of backends is obtained the order is non-deterministic:

``` python
    if _available_backends_list is None:
        _available_backends_list = [
            ep.resolve()
            for ep in pkg_resources.iter_entry_points(
                "cryptography.backends"
            )
        ]
```

The `iter_entry_points` documentation states "Note that within the entry points advertised by an individual distribution, there is no particular ordering". In Python 2.7 it does appear to provide consistent order, but in Py 3.x this is not true.

It's unclear to me why the OpenSSL backend is accepting a list of integers on Py3 though.

It sounds like the right solution is to perform a type check in
`GCM.__init__`

On Wed, Feb 10, 2016 at 3:45 PM, Paul Kehrer notifications@github.com
wrote:

> I can reproduce this. The reason it is intermittent on Py 3.5 is that when
> default_backend is called and the set of backends is obtained the order
> is non-deterministic:
> 
> ```
> if _available_backends_list is None:
>     _available_backends_list = [
>         ep.resolve()
>         for ep in pkg_resources.iter_entry_points(
>             "cryptography.backends"
>         )
>     ]
> ```
> 
> The iter_entry_points documentation states "Note that within the entry
> points advertised by an individual distribution, there is no particular
> ordering". In Python 2.7 it does appear to provide consistent order, but in
> Py 3.x this is not true.
> 
> It's unclear to me why the OpenSSL backend is accepting a list of integers
> on Py3 though.
> 
> —
> Reply to this email directly or view it on GitHub
> https://github.com/pyca/cryptography/issues/2720#issuecomment-182576260.

## 

"I disapprove of what you say, but I will defend to the death your right to
say it." -- Evelyn Beatrice Hall (summarizing Voltaire)
"The people's good is the highest law." -- Cicero
GPG Key fingerprint: 125F 5C67 DFE9 4084

@alex that will stop it, but I'd like to know why a list of integers for key/nonce doesn't cause errors in Python 3 in the OpenSSL backend.

We also probably need to think about how we want to order entry points (we have made assumptions in the past that it is ordered, e.g. inserting commoncrypto at the beginning of the list on OS X).

@reaperhulk on the passes where it doesn't fail (I think this means OpenSSL in your notes) everything actually works, so using legit values as lists it correctly decrypts/verifies the tag etc. 

If the context of how we found this issue is of any use; our tests are here https://github.com/ONSdigital/eq-survey-runner/tree/eg-69-harden-portal-integration/tests/app/authentication 

Okay, the reason this occurs is because cffi is smart. In the case of the OpenSSL backend when you pass a list of integers to cffi (in Python 3.5) when the type is something like `const unsigned char *` it knows that's basically a bytearray and handles it. The type for a similar function in the commoncrypto backend is `void *` which prevents cffi from doing fancy things like that.

Aligning the underlying implementation details of each backend is not possible, so we should just be more aggressive about type checks for key, IV, and tag. (Separately we should work on the non-deterministic behavior of backend ordering)


Resolving patch:
diff --git a/src/cryptography/hazmat/primitives/ciphers/modes.py b/src/cryptography/hazmat/primitives/ciphers/modes.py
--- a/src/cryptography/hazmat/primitives/ciphers/modes.py
+++ b/src/cryptography/hazmat/primitives/ciphers/modes.py
@@ -67,6 +67,9 @@ class CBC(object):
     name = "CBC"
 
     def __init__(self, initialization_vector):
+        if not isinstance(initialization_vector, bytes):
+            raise TypeError("initialization_vector must be bytes")
+
         self._initialization_vector = initialization_vector
 
     initialization_vector = utils.read_only_property("_initialization_vector")
@@ -87,6 +90,9 @@ class OFB(object):
     name = "OFB"
 
     def __init__(self, initialization_vector):
+        if not isinstance(initialization_vector, bytes):
+            raise TypeError("initialization_vector must be bytes")
+
         self._initialization_vector = initialization_vector
 
     initialization_vector = utils.read_only_property("_initialization_vector")
@@ -99,6 +105,9 @@ class CFB(object):
     name = "CFB"
 
     def __init__(self, initialization_vector):
+        if not isinstance(initialization_vector, bytes):
+            raise TypeError("initialization_vector must be bytes")
+
         self._initialization_vector = initialization_vector
 
     initialization_vector = utils.read_only_property("_initialization_vector")
@@ -111,6 +120,9 @@ class CFB8(object):
     name = "CFB8"
 
     def __init__(self, initialization_vector):
+        if not isinstance(initialization_vector, bytes):
+            raise TypeError("initialization_vector must be bytes")
+
         self._initialization_vector = initialization_vector
 
     initialization_vector = utils.read_only_property("_initialization_vector")
@@ -123,6 +135,9 @@ class CTR(object):
     name = "CTR"
 
     def __init__(self, nonce):
+        if not isinstance(nonce, bytes):
+            raise TypeError("nonce must be bytes")
+
         self._nonce = nonce
 
     nonce = utils.read_only_property("_nonce")
@@ -154,6 +169,12 @@ def __init__(self, initialization_vector, tag=None, min_tag_length=16):
                     min_tag_length)
             )
 
+        if not isinstance(initialization_vector, bytes):
+            raise TypeError("initialization_vector must be bytes")
+
+        if tag is not None and not isinstance(tag, bytes):
+            raise TypeError("tag must be bytes or None")
+
         self._initialization_vector = initialization_vector
         self._tag = tag
 


Test patch:
diff --git a/tests/hazmat/primitives/test_block.py b/tests/hazmat/primitives/test_block.py
--- a/tests/hazmat/primitives/test_block.py
+++ b/tests/hazmat/primitives/test_block.py
@@ -177,3 +177,33 @@ def test_ctr(self, backend):
                 modes.CTR(b"abc"),
                 backend,
             )
+
+
+class TestModesRequireBytes(object):
+    def test_cbc(self):
+        with pytest.raises(TypeError):
+            modes.CBC([1] * 16)
+
+    def test_cfb(self):
+        with pytest.raises(TypeError):
+            modes.CFB([1] * 16)
+
+    def test_cfb8(self):
+        with pytest.raises(TypeError):
+            modes.CFB8([1] * 16)
+
+    def test_ofb(self):
+        with pytest.raises(TypeError):
+            modes.OFB([1] * 16)
+
+    def test_ctr(self):
+        with pytest.raises(TypeError):
+            modes.CTR([1] * 16)
+
+    def test_gcm_iv(self):
+        with pytest.raises(TypeError):
+            modes.GCM([1] * 16)
+
+    def test_gcm_tag(self):
+        with pytest.raises(TypeError):
+            modes.GCM(b"\x00" * 16, [1] * 16)