Repository: xmlsec/python-xmlsec (language: python)

Issue:
Segmentation fault when encrypting using EncryptionType.CONTENT
I'm getting a `Segmentation fault: 11` error when I try to encrypt an XML document using `xmlsec.EncryptionType.CONTENT`. It works using `xmlsec.EncryptionType.ELEMENT` though.

Here's the code and the XML document.

```python
from lxml import etree
import xmlsec

namespaces = {
    'soap': 'http://schemas.xmlsoap.org/soap/envelope/'
}

manager = xmlsec.KeysManager()
key = xmlsec.Key.from_file('certificate.pem', xmlsec.KeyFormat.CERT_PEM, None)
manager.add_key(key)
template = etree.parse('document.xml').getroot()
enc_data = xmlsec.template.encrypted_data_create(
template, xmlsec.Transform.AES128, type=xmlsec.EncryptionType.CONTENT, ns='xenc')
xmlsec.template.encrypted_data_ensure_cipher_value(enc_data)
key_info = xmlsec.template.encrypted_data_ensure_key_info(enc_data, ns='dsig')
enc_key = xmlsec.template.add_encrypted_key(key_info, xmlsec.Transform.RSA_PKCS1)
xmlsec.template.encrypted_data_ensure_cipher_value(enc_key)
data = template.find('soap:Body', namespaces=namespaces)
enc_ctx = xmlsec.EncryptionContext(manager)
enc_ctx.key = xmlsec.Key.generate(xmlsec.KeyData.AES, 192, xmlsec.KeyDataType.SESSION)
enc_ctx.encrypt_xml(enc_data, data)
```

[document.xml.txt](https://github.com/mehcode/python-xmlsec/files/2214671/document.xml.txt)

And here's some `lldb` info I gathered:

```
Process 82119 stopped
* thread #1, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0x0)
    frame #0: 0x00000001085e23bc xmlsec.so`PyXmlSec_ClearReplacedNodes + 47
xmlsec.so`PyXmlSec_ClearReplacedNodes:
->  0x1085e23bc <+47>: decq   (%rax)
    0x1085e23bf <+50>: jne    0x1085e23e5               ; <+88>
    0x1085e23c1 <+52>: movq   %r15, %rdi
    0x1085e23c4 <+55>: movq   %rbx, %rsi
Target 0: (python) stopped.
(lldb) bt
* thread #1, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0x0)
  * frame #0: 0x00000001085e23bc xmlsec.so`PyXmlSec_ClearReplacedNodes + 47
    frame #1: 0x00000001085e1f6d xmlsec.so`PyXmlSec_EncryptionContextEncryptXml + 368
    frame #2: 0x0000000100112f2d Python`PyEval_EvalFrameEx + 10117
    frame #3: 0x00000001001105b6 Python`PyEval_EvalCodeEx + 1551
    frame #4: 0x00000001000b6d36 Python`function_call + 327
    frame #5: 0x00000001000993cd Python`PyObject_Call + 97
    frame #6: 0x0000000100115314 Python`PyEval_EvalFrameEx + 19308
    frame #7: 0x00000001001105b6 Python`PyEval_EvalCodeEx + 1551
    frame #8: 0x00000001000b6d36 Python`function_call + 327
    frame #9: 0x00000001000993cd Python`PyObject_Call + 97
    frame #10: 0x0000000100115314 Python`PyEval_EvalFrameEx + 19308
    frame #11: 0x00000001001105b6 Python`PyEval_EvalCodeEx + 1551
    frame #12: 0x0000000100118d65 Python`fast_function + 290
    frame #13: 0x0000000100112e6d Python`PyEval_EvalFrameEx + 9925
    frame #14: 0x00000001001105b6 Python`PyEval_EvalCodeEx + 1551
    frame #15: 0x0000000100118d65 Python`fast_function + 290
    frame #16: 0x0000000100112e6d Python`PyEval_EvalFrameEx + 9925
    frame #17: 0x00000001001105b6 Python`PyEval_EvalCodeEx + 1551
    frame #18: 0x0000000100118d65 Python`fast_function + 290
    frame #19: 0x0000000100112e6d Python`PyEval_EvalFrameEx + 9925
    frame #20: 0x00000001001105b6 Python`PyEval_EvalCodeEx + 1551
    frame #21: 0x000000010010ffa1 Python`PyEval_EvalCode + 32
    frame #22: 0x000000010013110b Python`run_mod + 49
    frame #23: 0x00000001001311b2 Python`PyRun_FileExFlags + 130
    frame #24: 0x0000000100130d34 Python`PyRun_SimpleFileExFlags + 706
    frame #25: 0x0000000100141dfe Python`Py_Main + 3030
    frame #26: 0x00007fffc4665235 libdyld.dylib`start + 1
    frame #27: 0x00007fffc4665235 libdyld.dylib`start + 1
```

## Environments

I was to reproduce in the following environments.

### macOS Sierra 10.12.6
- libxmlsec1 1.2.25 1.2.26
- libxml2 2.9.6 2.9.7
- xmlsec (1.3.3)

```
$ pkg-config --cflags xmlsec1
-D__XMLSEC_FUNCTION__=__func__ -DXMLSEC_NO_SIZE_T -DXMLSEC_NO_GOST=1 -DXMLSEC_NO_GOST2012=1 -DXMLSEC_NO_CRYPTO_DYNAMIC_LOADING=1 -DXMLSEC_CRYPTO_OPENSSL=1 -I/usr/local/Cellar/libxmlsec1/1.2.26/include/xmlsec1 -I/usr/local/opt/openssl/include -I/usr/local/opt/openssl/include/openssl -I/usr/include/libxml2
```

### Ubuntu 14.04.3 LTS

- libxml2:amd64 2.9.1+dfsg1-3ubuntu4.12
- libxmlsec1 1.2.18-2ubuntu1
- xmlsec 1.3.3       

```
$ pkg-config --cflags xmlsec1
-DXMLSEC_CRYPTO=\"openssl\" -D__XMLSEC_FUNCTION__=__FUNCTION__ -DXMLSEC_NO_SIZE_T -DXMLSEC_NO_GOST=1 -DXMLSEC_NO_XKMS=1 -DXMLSEC_NO_CRYPTO_DYNAMIC_LOADING=1 -DXMLSEC_OPENSSL_098=1 -DXMLSEC_CRYPTO_OPENSSL=1 -I/usr/include/libxml2 -I/usr/include/xmlsec1 
```




Discussion / hints:
@bgaifullin @mehcode Please let me know if there's anything else you need from me. Any tips are appreciated. Thanks in advance!
Thank you for bug report. I prepare the fix. #94 commit 1e46b9851fc7877cf6c6242ecc54dae764f0e986

after fix lxml reports an error:
```
Traceback (most recent call last):
  File "test.py", line 21, in <module>
    enc_ctx.encrypt_xml(enc_data, data)
  File "src/lxml/public-api.pxi", line 29, in lxml.etree.elementFactory
  File "src/lxml/etree.pyx", line 1605, in lxml.etree._elementFactory
  File "src/lxml/classlookup.pxi", line 405, in lxml.etree._parser_class_lookup
  File "src/lxml/classlookup.pxi", line 257, in lxml.etree._callLookupFallback
  File "src/lxml/classlookup.pxi", line 336, in lxml.etree._lookupDefaultElementClass
AssertionError: Unknown node type: 3
```
This shouldn't be an issue anymore since the fix from @bgaifullin was published in the latest release.

Fix patch:
diff --git a/pyproject.toml b/pyproject.toml
index 636c52c9..6c69db80 100644
--- a/pyproject.toml
+++ b/pyproject.toml
@@ -18,14 +18,7 @@ exclude = '''
 '''
 
 [tool.isort]
-force_alphabetical_sort_within_sections = true
-recursive = true
-line_length = 130
-multi_line_output = 3
-include_trailing_comma = true
-force_grid_wrap = 0
-use_parentheses = true
-combine_as_imports = true
+profile = 'black'
 known_first_party = ['xmlsec']
 known_third_party = ['lxml', 'pytest', '_pytest', 'hypothesis']
 
diff --git a/src/exception.c b/src/exception.c
index 2ca5ab57..176cf385 100644
--- a/src/exception.c
+++ b/src/exception.c
@@ -165,6 +165,12 @@ void PyXmlSecEnableDebugTrace(int v) {
     PyXmlSec_PrintErrorMessage = v;
 }
 
+void PyXmlSec_InstallErrorCallback() {
+    if (PyXmlSec_LastErrorKey != 0) {
+        xmlSecErrorsSetCallback(PyXmlSec_ErrorCallback);
+    }
+}
+
 // initializes errors module
 int PyXmlSec_ExceptionsModule_Init(PyObject* package) {
     PyXmlSec_Error = NULL;
@@ -185,9 +191,7 @@ int PyXmlSec_ExceptionsModule_Init(PyObject* package) {
     if (PyModule_AddObject(package, "VerificationError", PyXmlSec_VerificationError) < 0) goto ON_FAIL;
 
     PyXmlSec_LastErrorKey = PyThread_create_key();
-    if (PyXmlSec_LastErrorKey != 0) {
-        xmlSecErrorsSetCallback(&PyXmlSec_ErrorCallback);
-    }
+    PyXmlSec_InstallErrorCallback();
 
     return 0;
 
diff --git a/src/exception.h b/src/exception.h
index 9dea5ecb..687cd778 100644
--- a/src/exception.h
+++ b/src/exception.h
@@ -24,4 +24,6 @@ void PyXmlSec_ClearError(void);
 
 void PyXmlSecEnableDebugTrace(int);
 
+void PyXmlSec_InstallErrorCallback();
+
 #endif //__PYXMLSEC_EXCEPTIONS_H__
diff --git a/src/main.c b/src/main.c
index a602982c..c93b16d2 100644
--- a/src/main.c
+++ b/src/main.c
@@ -239,6 +239,11 @@ PYENTRY_FUNC_NAME(void)
 
     if (PyXmlSec_Init() < 0) goto ON_FAIL;
 
+    // xmlsec will install default callback in PyXmlSec_Init,
+    // overwriting any custom callbacks.
+    // We thus install our callback again now.
+    PyXmlSec_InstallErrorCallback();
+
     if (PyModule_AddStringConstant(module, "__version__", STRINGIFY(MODULE_VERSION)) < 0) goto ON_FAIL;
 
     if (PyXmlSec_InitLxmlModule() < 0) goto ON_FAIL;


Test patch:
diff --git a/tests/base.py b/tests/base.py
index cf659b61..e834f080 100644
--- a/tests/base.py
+++ b/tests/base.py
@@ -1,13 +1,13 @@
 import gc
 import os
 import sys
+import unittest
 
 from lxml import etree
-import xmlsec
 
-import unittest
+import xmlsec
 
-if sys.version_info < (3, ):
+if sys.version_info < (3,):
     unittest.TestCase.assertRaisesRegex = unittest.TestCase.assertRaisesRegexp
 
 
@@ -21,6 +21,8 @@
 
     def get_memory_usage():
         return resource.getrusage(resource.RUSAGE_SELF).ru_maxrss
+
+
 except ImportError:
     resource = None
 
@@ -112,9 +114,7 @@ def assertXmlEqual(self, first, second, msg=None):
             self.fail('Tags do not match: %s and %s. %s' % (first.tag, second.tag, msg))
         for name, value in first.attrib.items():
             if second.attrib.get(name) != value:
-                self.fail(
-                    'Attributes do not match: %s=%r, %s=%r. %s' % (name, value, name, second.attrib.get(name), msg)
-                )
+                self.fail('Attributes do not match: %s=%r, %s=%r. %s' % (name, value, name, second.attrib.get(name), msg))
         for name in second.attrib.keys():
             if name not in first.attrib:
                 self.fail('x2 has an attribute x1 is missing: %s. %s' % (name, msg))
diff --git a/tests/test_constants.py b/tests/test_constants.py
index 857a1cdd..689edce6 100644
--- a/tests/test_constants.py
+++ b/tests/test_constants.py
@@ -1,8 +1,9 @@
 """Test constants from :mod:`xmlsec.constants` module."""
 
-import xmlsec
 from hypothesis import given, strategies
 
+import xmlsec
+
 
 def _constants(typename):
     return list(
diff --git a/tests/test_ds.py b/tests/test_ds.py
index 98b6424b..9417fedb 100644
--- a/tests/test_ds.py
+++ b/tests/test_ds.py
@@ -1,4 +1,5 @@
 import unittest
+
 import xmlsec
 from tests import base
 
@@ -70,7 +71,7 @@ def test_sign_bad_args(self):
     def test_sign_fail(self):
         ctx = xmlsec.SignatureContext()
         ctx.key = xmlsec.Key.from_file(self.path("rsakey.pem"), format=consts.KeyDataFormatPem)
-        with self.assertRaisesRegex(xmlsec.InternalError, 'failed to sign'):
+        with self.assertRaisesRegex(xmlsec.Error, 'failed to sign'):
             ctx.sign(self.load_xml('sign1-in.xml'))
 
     def test_sign_case1(self):
@@ -229,7 +230,7 @@ def test_verify_bad_args(self):
     def test_verify_fail(self):
         ctx = xmlsec.SignatureContext()
         ctx.key = xmlsec.Key.from_file(self.path("rsakey.pem"), format=consts.KeyDataFormatPem)
-        with self.assertRaisesRegex(xmlsec.InternalError, 'failed to verify'):
+        with self.assertRaisesRegex(xmlsec.Error, 'failed to verify'):
             ctx.verify(self.load_xml('sign1-in.xml'))
 
     def test_verify_case_1(self):
diff --git a/tests/test_enc.py b/tests/test_enc.py
index 141e5753..7add848c 100644
--- a/tests/test_enc.py
+++ b/tests/test_enc.py
@@ -191,7 +191,7 @@ def test_encrypt_uri_bad_args(self):
 
     def test_encrypt_uri_fail(self):
         ctx = xmlsec.EncryptionContext()
-        with self.assertRaisesRegex(xmlsec.InternalError, 'failed to encrypt URI'):
+        with self.assertRaisesRegex(xmlsec.Error, 'failed to encrypt URI'):
             ctx.encrypt_uri(etree.Element('root'), '')
 
     def test_decrypt1(self):
diff --git a/tests/test_keys.py b/tests/test_keys.py
index 12b8224f..0d41abef 100644
--- a/tests/test_keys.py
+++ b/tests/test_keys.py
@@ -17,7 +17,7 @@ def test_key_from_memory_with_bad_args(self):
             xmlsec.Key.from_memory(1, format="")
 
     def test_key_from_memory_invalid_data(self):
-        with self.assertRaisesRegex(xmlsec.InternalError, '.*cannot load key.*'):
+        with self.assertRaisesRegex(xmlsec.Error, '.*cannot load key.*'):
             xmlsec.Key.from_memory(b'foo', format=consts.KeyDataFormatPem)
 
     def test_key_from_file(self):
@@ -29,7 +29,7 @@ def test_key_from_file_with_bad_args(self):
             xmlsec.Key.from_file(1, format="")
 
     def test_key_from_invalid_file(self):
-        with self.assertRaisesRegex(xmlsec.InternalError, '.*cannot read key.*'):
+        with self.assertRaisesRegex(xmlsec.Error, '.*cannot read key.*'):
             with tempfile.NamedTemporaryFile() as tmpfile:
                 tmpfile.write(b'foo')
                 xmlsec.Key.from_file(tmpfile.name, format=consts.KeyDataFormatPem)
@@ -42,7 +42,7 @@ def test_key_from_fileobj(self):
     def test_key_from_invalid_fileobj(self):
         with tempfile.NamedTemporaryFile(delete=False) as tmpfile:
             tmpfile.write(b'foo')
-        with self.assertRaisesRegex(xmlsec.InternalError, '.*cannot read key.*'), open(tmpfile.name) as fp:
+        with self.assertRaisesRegex(xmlsec.Error, '.*cannot read key.*'), open(tmpfile.name) as fp:
             xmlsec.Key.from_file(fp, format=consts.KeyDataFormatPem)
 
     def test_generate(self):
@@ -54,7 +54,7 @@ def test_generate_with_bad_args(self):
             xmlsec.Key.generate(klass="", size="", type="")
 
     def test_generate_invalid_size(self):
-        with self.assertRaisesRegex(xmlsec.InternalError, '.*cannot generate key.*'):
+        with self.assertRaisesRegex(xmlsec.Error, '.*cannot generate key.*'):
             xmlsec.Key.generate(klass=consts.KeyDataAes, size=0, type=consts.KeyDataTypeSession)
 
     def test_from_binary_file(self):
@@ -66,7 +66,7 @@ def test_from_binary_file_with_bad_args(self):
             xmlsec.Key.from_binary_file(klass="", filename=1)
 
     def test_from_invalid_binary_file(self):
-        with self.assertRaisesRegex(xmlsec.InternalError, '.*cannot read key.*'):
+        with self.assertRaisesRegex(xmlsec.Error, '.*cannot read key.*'):
             with tempfile.NamedTemporaryFile() as tmpfile:
                 tmpfile.write(b'foo')
                 xmlsec.Key.from_binary_file(klass=consts.KeyDataDes, filename=tmpfile.name)
@@ -80,7 +80,7 @@ def test_from_binary_data_with_bad_args(self):
             xmlsec.Key.from_binary_data(klass="", data=1)
 
     def test_from_invalid_binary_data(self):
-        with self.assertRaisesRegex(xmlsec.InternalError, '.*cannot read key.*'):
+        with self.assertRaisesRegex(xmlsec.Error, '.*cannot read key.*'):
             xmlsec.Key.from_binary_data(klass=consts.KeyDataDes, data=b'')
 
     def test_load_cert_from_file(self):
@@ -97,7 +97,7 @@ def test_load_cert_from_file_with_bad_args(self):
     def test_load_cert_from_invalid_file(self):
         key = xmlsec.Key.from_file(self.path("rsakey.pem"), format=consts.KeyDataFormatPem)
         self.assertIsNotNone(key)
-        with self.assertRaisesRegex(xmlsec.InternalError, '.*cannot load cert.*'):
+        with self.assertRaisesRegex(xmlsec.Error, '.*cannot load cert.*'):
             with tempfile.NamedTemporaryFile() as tmpfile:
                 tmpfile.write(b'foo')
                 key.load_cert_from_file(tmpfile.name, format=consts.KeyDataFormatPem)
@@ -119,7 +119,7 @@ def test_load_cert_from_invalid_fileobj(self):
         self.assertIsNotNone(key)
         with tempfile.NamedTemporaryFile(delete=False) as tmpfile:
             tmpfile.write(b'foo')
-        with self.assertRaisesRegex(xmlsec.InternalError, '.*cannot load cert.*'), open(tmpfile.name) as fp:
+        with self.assertRaisesRegex(xmlsec.Error, '.*cannot load cert.*'), open(tmpfile.name) as fp:
             key.load_cert_from_file(fp, format=consts.KeyDataFormatPem)
 
     def test_load_cert_from_memory(self):
@@ -136,7 +136,7 @@ def test_load_cert_from_memory_with_bad_args(self):
     def test_load_cert_from_memory_invalid_data(self):
         key = xmlsec.Key.from_file(self.path("rsakey.pem"), format=consts.KeyDataFormatPem)
         self.assertIsNotNone(key)
-        with self.assertRaisesRegex(xmlsec.InternalError, '.*cannot load cert.*'):
+        with self.assertRaisesRegex(xmlsec.Error, '.*cannot load cert.*'):
             key.load_cert_from_memory(b'', format=consts.KeyDataFormatPem)
 
     def test_get_name(self):
@@ -190,7 +190,7 @@ def test_load_cert(self):
     def test_load_cert_with_bad_args(self):
         mngr = xmlsec.KeysManager()
         mngr.add_key(xmlsec.Key.from_file(self.path("rsakey.pem"), format=consts.KeyDataFormatPem))
-        with self.assertRaisesRegex(xmlsec.InternalError, '.*cannot load cert.*'):
+        with self.assertRaisesRegex(xmlsec.Error, '.*cannot load cert.*'):
             with tempfile.NamedTemporaryFile() as tmpfile:
                 tmpfile.write(b'foo')
                 mngr.load_cert(tmpfile.name, format=consts.KeyDataFormatPem, type=consts.KeyDataTypeTrusted)
@@ -215,7 +215,7 @@ def test_load_cert_from_memory_with_bad_args(self):
     def test_load_cert_from_memory_invalid_data(self):
         mngr = xmlsec.KeysManager()
         mngr.add_key(xmlsec.Key.from_file(self.path("rsakey.pem"), format=consts.KeyDataFormatPem))
-        with self.assertRaisesRegex(xmlsec.InternalError, '.*cannot load cert.*'):
+        with self.assertRaisesRegex(xmlsec.Error, '.*cannot load cert.*'):
             mngr.load_cert_from_memory(b'', format=consts.KeyDataFormatPem, type=consts.KeyDataTypeTrusted)
 
     def test_load_invalid_key(self):
diff --git a/tests/test_templates.py b/tests/test_templates.py
index 9475c5e4..3bae7e55 100644
--- a/tests/test_templates.py
+++ b/tests/test_templates.py
@@ -1,4 +1,5 @@
 import unittest
+
 from lxml import etree
 
 import xmlsec
@@ -36,7 +37,7 @@ def test_ensure_key_info(self):
         self.assertEqual("Id", ki.get("Id"))
 
     def test_ensure_key_info_fail(self):
-        with self.assertRaisesRegex(xmlsec.InternalError, 'cannot ensure key info.'):
+        with self.assertRaisesRegex(xmlsec.Error, 'cannot ensure key info.'):
             xmlsec.template.ensure_key_info(etree.fromstring(b'<Data/>'), id="Id")
 
     def test_ensure_key_info_bad_args(self):
@@ -88,7 +89,7 @@ def test_add_reference_bad_args(self):
             xmlsec.template.add_reference(etree.Element('root'), '')
 
     def test_add_reference_fail(self):
-        with self.assertRaisesRegex(xmlsec.InternalError, 'cannot add reference.'):
+        with self.assertRaisesRegex(xmlsec.Error, 'cannot add reference.'):
             xmlsec.template.add_reference(etree.Element('root'), consts.TransformSha1)
 
     def test_add_transform_bad_args(self):
diff --git a/tests/test_xmlsec.py b/tests/test_xmlsec.py
index 32fac69a..2d470a11 100644
--- a/tests/test_xmlsec.py
+++ b/tests/test_xmlsec.py
@@ -10,5 +10,5 @@ def test_reinitialize_module(self):
         tests don't fail, we know that the ``init()``/``shutdown()``
         function pair doesn't break anything.
         """
-        xmlsec.shutdown()
-        xmlsec.init()
+        # xmlsec.shutdown()
+        # xmlsec.init()