Repository: pyca/cryptography

Issue:
Memory leak in X509v3 extension handler
Original report was https://github.com/pyca/pyopenssl/issues/825

I'm able to reproduce the memory leak with this script:

## leak script

```
CERT=b"""
-----BEGIN CERTIFICATE-----
MIIGLzCCBRegAwIBAgISAx8khDB8m8URsxI8sjakgNRRMA0GCSqGSIb3DQEBCwUA
MEoxCzAJBgNVBAYTAlVTMRYwFAYDVQQKEw1MZXQncyBFbmNyeXB0MSMwIQYDVQQD
ExpMZXQncyBFbmNyeXB0IEF1dGhvcml0eSBYMzAeFw0xODAzMjkxNzQ1MDdaFw0x
ODA2MjcxNzQ1MDdaMC0xKzApBgNVBAMTIjVjOWE3Zi5sZS10ZXN0LmhvZmZtYW4t
YW5kcmV3cy5jb20wggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQC86uj1
BNnZH8/GnblDJUp/7XxqPATi1cfd0BDLvFVYhydEicpPQy3ObXq4PQ1720nEZvvK
kxAtxj4OsfsqDFBlT9kLgabLNX9Y4m5Q91K/e/6bVhkBJqR0CYFPWVg73TN9+4ko
O+Iugebc4TtOIfpgCfyKf5A6F6sFyL7YWnFTVoN+hJ5XGWComZcB6unOBUTqq5Nr
eQw8NcN12xjpqmJ9X6NXmg+1+AeeSlyb4xwrkafzpjr9/tub1OpmaJAkF9KGvku+
XkPNn+G4lUwG8h9cVZT9OrfXqcvWq/GXdNZS/TXFcYwlo7oZZ4Rs7XDNupWDHPHg
n/e4AU5jAwznp3Z1AgMBAAGjggMqMIIDJjAOBgNVHQ8BAf8EBAMCBaAwHQYDVR0l
BBYwFAYIKwYBBQUHAwEGCCsGAQUFBwMCMAwGA1UdEwEB/wQCMAAwHQYDVR0OBBYE
FIs6Iaut9QxLMNzNgick0sS5uinjMB8GA1UdIwQYMBaAFKhKamMEfd265tE5t6ZF
Ze/zqOyhMG8GCCsGAQUFBwEBBGMwYTAuBggrBgEFBQcwAYYiaHR0cDovL29jc3Au
aW50LXgzLmxldHNlbmNyeXB0Lm9yZzAvBggrBgEFBQcwAoYjaHR0cDovL2NlcnQu
aW50LXgzLmxldHNlbmNyeXB0Lm9yZy8wLQYDVR0RBCYwJIIiNWM5YTdmLmxlLXRl
c3QuaG9mZm1hbi1hbmRyZXdzLmNvbTCB/gYDVR0gBIH2MIHzMAgGBmeBDAECATCB
5gYLKwYBBAGC3xMBAQEwgdYwJgYIKwYBBQUHAgEWGmh0dHA6Ly9jcHMubGV0c2Vu
Y3J5cHQub3JnMIGrBggrBgEFBQcCAjCBngyBm1RoaXMgQ2VydGlmaWNhdGUgbWF5
IG9ubHkgYmUgcmVsaWVkIHVwb24gYnkgUmVseWluZyBQYXJ0aWVzIGFuZCBvbmx5
IGluIGFjY29yZGFuY2Ugd2l0aCB0aGUgQ2VydGlmaWNhdGUgUG9saWN5IGZvdW5k
IGF0IGh0dHBzOi8vbGV0c2VuY3J5cHQub3JnL3JlcG9zaXRvcnkvMIIBBAYKKwYB
BAHWeQIEAgSB9QSB8gDwAHUA23Sv7ssp7LH+yj5xbSzluaq7NveEcYPHXZ1PN7Yf
v2QAAAFicxPrGQAABAMARjBEAiB+H80emivSpQoMgecTAzoHYjQNqPke8npIs4F2
QBWc0wIgZZ/p8diA4uj2syW+nxiVbRfGyopvKxLLD1X7cPdZpBkAdwApPFGWVMg5
ZbqqUPxYB9S3b79Yeily3KTDDPTlRUf0eAAAAWJzE+sqAAAEAwBIMEYCIQCrcvHk
1iI++H/GhJHCCNKdTVfr9HWIu3VE0y+VN+LOwQIhAIr/xAzGxOOyRXja3k+BXsvO
LVeleTQhGaHmW8fl5pziMA0GCSqGSIb3DQEBCwUAA4IBAQCV+HtmMXZ3ZQL3kt3S
MsIWlDx4A4dvy+tGOTo2NUlYE0SC4K/u05ARYYMnwvAgM1F1j+tCC3PObHl7mPiA
dvQJ85A/ND0fXZVA9B70frOb1htihzpE8At8i1k8akFkWM9LUxjzUjW8iOq7qjTz
4/gb07BH6YLuE2OIXoT3by8Hnytu607LWO/nTI3n1U3lyJxPtbsGlLg3vW8Cuv1a
bAB9G5PSUAe9qbK9v4IgH+G3a2KM404tl06OYj7Fely1O0Nd1LmZOt9ro5cvKynS
WVlKlOF7vgbzSq5c8PUCl1SMTf/FVmE294o9OzJOrpMaFOtr5todU45Izwd1g8Z7
Uufo
-----END CERTIFICATE-----
"""

import os
import psutil
from cryptography.x509 import load_pem_x509_certificate
from cryptography.hazmat.backends import default_backend
from cryptography.hazmat.backends.openssl import decode_asn1

p = psutil.Process(os.getpid())
for i in range(20000):
    if i % 2000 == 0:
        print(p.memory_info())
    c = load_pem_x509_certificate(CERT, default_backend())
    c.extensions[0]
```

## output

```
python3 leak.py 
pmem(rss=24264704, vms=251703296, shared=11440128, text=8192, lib=0, data=12877824, dirty=0)
pmem(rss=25075712, vms=252379136, shared=11853824, text=8192, lib=0, data=13553664, dirty=0)
pmem(rss=25886720, vms=252919808, shared=11853824, text=8192, lib=0, data=14094336, dirty=0)
pmem(rss=26427392, vms=253460480, shared=11853824, text=8192, lib=0, data=14635008, dirty=0)
pmem(rss=26968064, vms=254136320, shared=11853824, text=8192, lib=0, data=15310848, dirty=0)
pmem(rss=27508736, vms=254676992, shared=11853824, text=8192, lib=0, data=15851520, dirty=0)
pmem(rss=28049408, vms=255217664, shared=11853824, text=8192, lib=0, data=16392192, dirty=0)
pmem(rss=28590080, vms=255758336, shared=11853824, text=8192, lib=0, data=16932864, dirty=0)
pmem(rss=29401088, vms=256434176, shared=11853824, text=8192, lib=0, data=17608704, dirty=0)
pmem(rss=29941760, vms=256974848, shared=11853824, text=8192, lib=0, data=18149376, dirty=0)

```

## analysis

valgrind points to ``ASN1_item_d2i`` inside the v3 extension handler at line ``v3_lib.c:142``. The line is only used for some extension handlers. SCT extensions don't have ``method->it``. My bet is on ``v3_cpols``.

```
==10866== 36,600 (6,400 direct, 30,200 indirect) bytes in 400 blocks are definitely lost in loss record 5,886 of 5,889
==10866==    at 0x483880B: malloc (vg_replace_malloc.c:309)
==10866==    by 0x13EA1EBD: CRYPTO_malloc (mem.c:222)
==10866==    by 0x13EA1EF0: CRYPTO_zalloc (mem.c:230)
==10866==    by 0x13D7A6EB: asn1_item_embed_new (tasn_new.c:122)
==10866==    by 0x13D7A442: ASN1_item_ex_new (tasn_new.c:39)
==10866==    by 0x13D76DCD: asn1_item_embed_d2i (tasn_dec.c:306)
==10866==    by 0x13D77803: asn1_template_noexp_d2i (tasn_dec.c:592)
==10866==    by 0x13D77551: asn1_template_ex_d2i (tasn_dec.c:499)
==10866==    by 0x13D7674D: asn1_item_embed_d2i (tasn_dec.c:178)
==10866==    by 0x13D76597: ASN1_item_ex_d2i (tasn_dec.c:124)
==10866==    by 0x13D7652B: ASN1_item_d2i (tasn_dec.c:114)
==10866==    by 0x13F54867: X509V3_EXT_d2i (v3_lib.c:142)
==10866==    by 0x158A93EA: _cffi_f_X509V3_EXT_d2i (_openssl.c:43159)
```

```
(gdb) b crypto/x509v3/v3_lib.c:142
Breakpoint 1, X509V3_EXT_d2i (ext=0x5555557e3100) at crypto/x509v3/v3_lib.c:142
142             return ASN1_item_d2i(NULL, &p, extlen, ASN1_ITEM_ptr(method->it));
(gdb) p p
$1 = (const unsigned char *) 0x5555557e3680 "\003\002\005\240"
(gdb) p method
$2 = (const X509V3_EXT_METHOD *) 0x7fffea2a8480 <v3_key_usage>
(gdb) c
Continuing.

Breakpoint 1, X509V3_EXT_d2i (ext=0x5555557e3070) at crypto/x509v3/v3_lib.c:142
142             return ASN1_item_d2i(NULL, &p, extlen, ASN1_ITEM_ptr(method->it));
(gdb) p method
$3 = (const X509V3_EXT_METHOD *) 0x7fffea2a8c00 <v3_ext_ku>
(gdb) c
Continuing.

Breakpoint 1, X509V3_EXT_d2i (ext=0x5555557e31d0) at crypto/x509v3/v3_lib.c:142
142             return ASN1_item_d2i(NULL, &p, extlen, ASN1_ITEM_ptr(method->it));
(gdb) p method
$4 = (const X509V3_EXT_METHOD *) 0x7fffea2a8320 <v3_bcons>
(gdb) c
Continuing.

Breakpoint 1, X509V3_EXT_d2i (ext=0x5555557e3130) at crypto/x509v3/v3_lib.c:142
142             return ASN1_item_d2i(NULL, &p, extlen, ASN1_ITEM_ptr(method->it));
(gdb) p method
$5 = (const X509V3_EXT_METHOD *) 0x7fffea2a9ba0 <v3_skey_id>
(gdb) c
Continuing.

Breakpoint 1, X509V3_EXT_d2i (ext=0x5555557e3040) at crypto/x509v3/v3_lib.c:142
142             return ASN1_item_d2i(NULL, &p, extlen, ASN1_ITEM_ptr(method->it));
(gdb) p method
$6 = (const X509V3_EXT_METHOD *) 0x7fffea2a7ee0 <v3_akey_id>
(gdb) c
Continuing.

Breakpoint 1, X509V3_EXT_d2i (ext=0x5555557e3290) at crypto/x509v3/v3_lib.c:142
142             return ASN1_item_d2i(NULL, &p, extlen, ASN1_ITEM_ptr(method->it));
(gdb) c
Continuing.

Breakpoint 1, X509V3_EXT_d2i (ext=0x5555557e3230) at crypto/x509v3/v3_lib.c:142
142             return ASN1_item_d2i(NULL, &p, extlen, ASN1_ITEM_ptr(method->it));
(gdb) p method
$7 = (const X509V3_EXT_METHOD *) 0x7fffea2a7fe0 <v3_alt>
(gdb) c
Continuing.

Breakpoint 1, X509V3_EXT_d2i (ext=0x5555557e3320) at crypto/x509v3/v3_lib.c:142
142             return ASN1_item_d2i(NULL, &p, extlen, ASN1_ITEM_ptr(method->it));
(gdb) p method
$8 = (const X509V3_EXT_METHOD *) 0x7fffea2a8500 <v3_cpols>
(gdb) c
Continuing.
[Inferior 1 (process 11491) exited normally]
```


Discussion:
The leak is in AIA parsing for this cert. Fix in #4836, but that only fixes parsing. Generation appears to leak as well and applying the same fix doesn't actually completely resolve it, so more investigation is required.
Is this test case still leaking, @tiran? If not, I'd like to close this and open appropriate follow up bugs.

Resolving patch:
diff --git a/src/cryptography/hazmat/backends/openssl/encode_asn1.py b/src/cryptography/hazmat/backends/openssl/encode_asn1.py
--- a/src/cryptography/hazmat/backends/openssl/encode_asn1.py
+++ b/src/cryptography/hazmat/backends/openssl/encode_asn1.py
@@ -345,16 +345,22 @@ def _encode_authority_information_access(backend, authority_info_access):
     aia = backend._lib.sk_ACCESS_DESCRIPTION_new_null()
     backend.openssl_assert(aia != backend._ffi.NULL)
     aia = backend._ffi.gc(
-        aia, backend._lib.sk_ACCESS_DESCRIPTION_free
+        aia,
+        lambda x: backend._lib.sk_ACCESS_DESCRIPTION_pop_free(
+            x, backend._ffi.addressof(
+                backend._lib._original_lib, "ACCESS_DESCRIPTION_free"
+            )
+        )
     )
     for access_description in authority_info_access:
         ad = backend._lib.ACCESS_DESCRIPTION_new()
         method = _txt2obj(
             backend, access_description.access_method.dotted_string
         )
-        gn = _encode_general_name(backend, access_description.access_location)
+        _encode_general_name_preallocated(
+            backend, access_description.access_location, ad.location
+        )
         ad.method = method
-        ad.location = gn
         res = backend._lib.sk_ACCESS_DESCRIPTION_push(aia, ad)
         backend.openssl_assert(res >= 1)
 
@@ -385,8 +391,13 @@ def _encode_subject_key_identifier(backend, ski):
 
 
 def _encode_general_name(backend, name):
+    gn = backend._lib.GENERAL_NAME_new()
+    _encode_general_name_preallocated(backend, name, gn)
+    return gn
+
+
+def _encode_general_name_preallocated(backend, name, gn):
     if isinstance(name, x509.DNSName):
-        gn = backend._lib.GENERAL_NAME_new()
         backend.openssl_assert(gn != backend._ffi.NULL)
         gn.type = backend._lib.GEN_DNS
 
@@ -400,7 +411,6 @@ def _encode_general_name(backend, name):
         backend.openssl_assert(res == 1)
         gn.d.dNSName = ia5
     elif isinstance(name, x509.RegisteredID):
-        gn = backend._lib.GENERAL_NAME_new()
         backend.openssl_assert(gn != backend._ffi.NULL)
         gn.type = backend._lib.GEN_RID
         obj = backend._lib.OBJ_txt2obj(
@@ -409,13 +419,11 @@ def _encode_general_name(backend, name):
         backend.openssl_assert(obj != backend._ffi.NULL)
         gn.d.registeredID = obj
     elif isinstance(name, x509.DirectoryName):
-        gn = backend._lib.GENERAL_NAME_new()
         backend.openssl_assert(gn != backend._ffi.NULL)
         dir_name = _encode_name(backend, name.value)
         gn.type = backend._lib.GEN_DIRNAME
         gn.d.directoryName = dir_name
     elif isinstance(name, x509.IPAddress):
-        gn = backend._lib.GENERAL_NAME_new()
         backend.openssl_assert(gn != backend._ffi.NULL)
         if isinstance(name.value, ipaddress.IPv4Network):
             packed = (
@@ -433,7 +441,6 @@ def _encode_general_name(backend, name):
         gn.type = backend._lib.GEN_IPADD
         gn.d.iPAddress = ipaddr
     elif isinstance(name, x509.OtherName):
-        gn = backend._lib.GENERAL_NAME_new()
         backend.openssl_assert(gn != backend._ffi.NULL)
         other_name = backend._lib.OTHERNAME_new()
         backend.openssl_assert(other_name != backend._ffi.NULL)
@@ -456,7 +463,6 @@ def _encode_general_name(backend, name):
         gn.type = backend._lib.GEN_OTHERNAME
         gn.d.otherName = other_name
     elif isinstance(name, x509.RFC822Name):
-        gn = backend._lib.GENERAL_NAME_new()
         backend.openssl_assert(gn != backend._ffi.NULL)
         # ia5strings are supposed to be ITU T.50 but to allow round-tripping
         # of broken certs that encode utf8 we'll encode utf8 here too.
@@ -465,7 +471,6 @@ def _encode_general_name(backend, name):
         gn.type = backend._lib.GEN_EMAIL
         gn.d.rfc822Name = asn1_str
     elif isinstance(name, x509.UniformResourceIdentifier):
-        gn = backend._lib.GENERAL_NAME_new()
         backend.openssl_assert(gn != backend._ffi.NULL)
         # ia5strings are supposed to be ITU T.50 but to allow round-tripping
         # of broken certs that encode utf8 we'll encode utf8 here too.
@@ -478,8 +483,6 @@ def _encode_general_name(backend, name):
             "{} is an unknown GeneralName type".format(name)
         )
 
-    return gn
-
 
 def _encode_extended_key_usage(backend, extended_key_usage):
     eku = backend._lib.sk_ASN1_OBJECT_new_null()


Test patch:
diff --git a/tests/hazmat/backends/test_openssl_memleak.py b/tests/hazmat/backends/test_openssl_memleak.py
--- a/tests/hazmat/backends/test_openssl_memleak.py
+++ b/tests/hazmat/backends/test_openssl_memleak.py
@@ -389,3 +389,63 @@ def func():
                 x509.IssuingDistributionPoint
             )
         """))
+
+    def test_create_certificate_with_extensions(self):
+        assert_no_memory_leaks(textwrap.dedent("""
+        def func():
+            import datetime
+
+            from cryptography import x509
+            from cryptography.hazmat.backends.openssl import backend
+            from cryptography.hazmat.primitives import hashes
+            from cryptography.hazmat.primitives.asymmetric import ec
+            from cryptography.x509.oid import (
+                AuthorityInformationAccessOID, ExtendedKeyUsageOID, NameOID
+            )
+
+            private_key = ec.generate_private_key(ec.SECP256R1(), backend)
+
+            not_valid_before = datetime.datetime.now()
+            not_valid_after = not_valid_before + datetime.timedelta(days=365)
+
+            aia = x509.AuthorityInformationAccess([
+                x509.AccessDescription(
+                    AuthorityInformationAccessOID.OCSP,
+                    x509.UniformResourceIdentifier(u"http://ocsp.domain.com")
+                ),
+                x509.AccessDescription(
+                    AuthorityInformationAccessOID.CA_ISSUERS,
+                    x509.UniformResourceIdentifier(u"http://domain.com/ca.crt")
+                )
+            ])
+            sans = [u'*.example.org', u'foobar.example.net']
+            san = x509.SubjectAlternativeName(list(map(x509.DNSName, sans)))
+
+            ski = x509.SubjectKeyIdentifier.from_public_key(
+                private_key.public_key()
+            )
+            eku = x509.ExtendedKeyUsage([
+                ExtendedKeyUsageOID.CLIENT_AUTH,
+                ExtendedKeyUsageOID.SERVER_AUTH,
+                ExtendedKeyUsageOID.CODE_SIGNING,
+            ])
+
+            builder = x509.CertificateBuilder().serial_number(
+                777
+            ).issuer_name(x509.Name([
+                x509.NameAttribute(NameOID.COUNTRY_NAME, u'US'),
+            ])).subject_name(x509.Name([
+                x509.NameAttribute(NameOID.COUNTRY_NAME, u'US'),
+            ])).public_key(
+                private_key.public_key()
+            ).add_extension(
+                aia, critical=False
+            ).not_valid_before(
+                not_valid_before
+            ).not_valid_after(
+                not_valid_after
+            )
+
+            cert = builder.sign(private_key, hashes.SHA256(), backend)
+            cert.extensions
+        """))