Repository: xmlsec/python-xmlsec (language: python)

Issue:
Support PKCS11
I'm not aware about any way how to use PKCS11 device (token, hsm) with this library. This is critical feature for many use cases. 


Discussion / hints:
NSS crypto engine in xmlsec should be capable of using PKCS11. Is it possible to switch to NSS crypto with python-xmlsec? 
@jtalir first thing I would check is whether you can sign/verify or encrypt/decrypt the document with just `xmlsec1` - there's the command line tool for that. As for building against `xmlsec1-nss` - never done that, but I suppose this should be possible once the constants that are not supported by the NSS engine are sorted out in `constants.c`. I will create an experimental branch you can build the package from, so you can try it out locally.
@jtalir try out the `nss-crypto` branch:
```sh
$ PYXMLSEC_CRYPTO_ENGINE=nss pip install git+https://github.com/hoefling/xmlsec.git@nss-crypto
```
should build against `xmlsec1-nss`. You need to install the lib first - on my Fedora 33, it's `dnf install xmlsec1-nss xmlsec1-nss-devel`.
Thanks. I was able to build it. Any hint where to provide content of  --crypto-config  parametr from CLI xmlsec1?
@jtalir this is yet missing :-) Let me add a sample impl first.
@jtalir try reinstalling from the same branch again. I have hardcoded `nss` as crypto engine for the sake of testing, otherwise `xmlsec1` overwrites it for me - will need a better way of passing the crypto engine way than what's implemented currently.
@jtalir you can pass the config path via `xmlsec.init`:
```py
import xmlsec

xmlsec.init('path/to/ssl.conf')

...
```
If it doesn't work, try a `shutdown` before reinitialization:
```py
import xmlsec

xmlsec.shutdown()
xmlsec.init('path/to/ssl.conf')
```
No success. Trying simple nss with local keystore (no pkcs11). Simple xmlsec1 call works, python script doesnt. See bellow:
```
[/tmp/test-nss]$ cat test-mykey.xml 
<RootElement>
 <Signature xmlns="http://www.w3.org/2000/09/xmldsig#">
  <SignedInfo>
   <CanonicalizationMethod Algorithm=
      "http://www.w3.org/TR/2001/REC-xml-c14n-20010315"/>
    <SignatureMethod Algorithm=
      "http://www.w3.org/2000/09/xmldsig#rsa-sha1"/>
     <Reference URI="">
      <Transforms>
       <Transform Algorithm=
         "http://www.w3.org/2000/09/xmldsig#enveloped-signature" />
      </Transforms>
       <DigestMethod Algorithm=
          "http://www.w3.org/2000/09/xmldsig#sha1"/>
        <DigestValue></DigestValue>
     </Reference>
  </SignedInfo>
  <SignatureValue />
  <KeyInfo>
    <KeyName>mykey</KeyName>
  </KeyInfo>
 </Signature>
</RootElement>
[/tmp/test-nss]$ xmlsec1 --sign --crypto nss --crypto-config nss test-mykey.xml
<?xml version="1.0"?>
<RootElement>
 <Signature xmlns="http://www.w3.org/2000/09/xmldsig#">
  <SignedInfo>
   <CanonicalizationMethod Algorithm="http://www.w3.org/TR/2001/REC-xml-c14n-20010315"/>
    <SignatureMethod Algorithm="http://www.w3.org/2000/09/xmldsig#rsa-sha1"/>
     <Reference URI="">
      <Transforms>
       <Transform Algorithm="http://www.w3.org/2000/09/xmldsig#enveloped-signature"/>
      </Transforms>
       <DigestMethod Algorithm="http://www.w3.org/2000/09/xmldsig#sha1"/>
        <DigestValue>4uafMBC7xp0uZh7G+Pwl6OTlRLk=</DigestValue>
     </Reference>
  </SignedInfo>
  <SignatureValue>KX2IQezSULSOzPRSLEDW5Ra1xx5fq01VWsXClu627lOrXlJKZEwTNFj/F4JmxJ+C
v8Qk1h2gB57x8vFzWUEAZJe6APW6W7rkyRfYZR4UlqO+L6lfycdp9PwpEKyrtwLg
g2kWsd9f/VuzAMfoFTq45WQg0D4QizRSObXoqutSNWtO0KnfQVPLqMkh4ENU1iht
gCUdrA3BAO/qy9v9Ts/3WFB6QCcAdePk3ofeaN4WQyNP4MRx82e5eSJTz58IDoel
urT/bA0T7PHyBaeokP9ve4ngSl/BPWC+0cng1DrFO5GZlxmgPRBJt7X/oGttUSAt
R+2XhQoZxtmgU3hCxl5Psw==</SignatureValue>
  <KeyInfo>
    <KeyName>mykey</KeyName>
  </KeyInfo>
 </Signature>
</RootElement>
[/tmp/test-nss]$ PYXMLSEC_CRYPTO_ENGINE=nss pip install --target=/tmp/xmlsec/ git+https://github.com/hoefling/xmlsec.git@nss-crypto
Collecting git+https://github.com/hoefling/xmlsec.git@nss-crypto
  Cloning https://github.com/hoefling/xmlsec.git (to revision nss-crypto) to /tmp/pip-req-build-fku86ufe
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Installing backend dependencies ... done
    Preparing wheel metadata ... done
Collecting lxml>=3.8
  Using cached lxml-4.6.2-cp39-cp39-manylinux1_x86_64.whl (5.4 MB)
Building wheels for collected packages: xmlsec
  Building wheel for xmlsec (PEP 517) ... done
  Created wheel for xmlsec: filename=xmlsec-1.3.8.dev31+g3531b39-cp39-cp39-linux_x86_64.whl size=150210 sha256=92f6564c121e7dc79f83291a93faae8e1ab32cecda2a42f92cf1f9296c2a26bc
  Stored in directory: /tmp/pip-ephem-wheel-cache-az21zazg/wheels/07/74/18/199e74712eb369fa5def70e1f8c05ddaad1e4c721bb36f096f
Successfully built xmlsec
Installing collected packages: lxml, xmlsec
Successfully installed lxml-4.6.2 xmlsec-1.3.8.dev31+g3531b39
[/tmp/test-nss]$ cat sign.py 
from lxml import etree
import xmlsec
xmlsec.shutdown()
xmlsec.init('./nss/')
template = etree.parse('test-mykey.xml').getroot()
signature_node = xmlsec.tree.find_node(template, xmlsec.constants.NodeSignature)
ctx = xmlsec.SignatureContext()
ctx.sign(signature_node)
print(etree.tostring(template))
[/tmp/test-nss]$ PYTHONPATH=/tmp/xmlsec/ python sign.py 
func=xmlSecDSigCtxProcessKeyInfoNode:file=xmldsig.c:line=793:obj=unknown:subj=unknown:error=45:key is not found:details=NULL
func=xmlSecDSigCtxProcessSignatureNode:file=xmldsig.c:line=508:obj=unknown:subj=xmlSecDSigCtxProcessKeyInfoNode:error=1:xmlsec library function failed: 
func=xmlSecDSigCtxSign:file=xmldsig.c:line=291:obj=unknown:subj=xmlSecDSigCtxProcessSignatureNode:error=1:xmlsec library function failed: 
Traceback (most recent call last):
  File "/tmp/test-nss/sign.py", line 8, in <module>
    ctx.sign(signature_node)
xmlsec.InternalError: (-1, 'failed to sign')
```

Maybe it's related to the fact that in nss crypto engine, key is identified by name in template.
There is recent release of xmlsec-1.2.33 there is a new attribute --privkey-openssl-engine that can pass pkcs11 parameters to openssl engine as discussed in this thread https://www.aleksey.com/pipermail/xmlsec/2021/010434.html.

How hard is it to get this through python-xmlsec to be able to use it the same way as command line client?
Is anybody able to evaluate if this change in xmlsec-1.2.33 is callable via python-xmlsec?

Fix patch:
diff --git a/.github/workflows/sdist.yml b/.github/workflows/sdist.yml
index d7959208..6ca7a657 100644
--- a/.github/workflows/sdist.yml
+++ b/.github/workflows/sdist.yml
@@ -16,9 +16,8 @@ jobs:
         run: |
           python setup.py sdist
       - name: Install test dependencies
-        env:
-          PYXMLSEC_STATIC_DEPS: true
         run: |
+          sudo apt-get install libxml2-dev libxmlsec1-dev libxmlsec1-openssl opensc softhsm2 libengine-pkcs11-openssl
           pip install --upgrade -r requirements-test.txt
           pip install black  # for stub generation tests
           pip install dist/xmlsec-$(python setup.py --version).tar.gz
diff --git a/README.rst b/README.rst
index 34bdd377..e1924652 100644
--- a/README.rst
+++ b/README.rst
@@ -37,7 +37,7 @@ Check the `examples <https://xmlsec.readthedocs.io/en/latest/examples.html>`_ se
 Requirements
 ************
 - ``libxml2 >= 2.9.1``
-- ``libxmlsec1 >= 1.2.18``
+- ``libxmlsec1 >= 1.2.33``
 
 Install
 *******
diff --git a/src/keys.c b/src/keys.c
index 47678a53..a8ffde25 100644
--- a/src/keys.c
+++ b/src/keys.c
@@ -185,6 +185,47 @@ static PyObject* PyXmlSec_KeyFromFile(PyObject* self, PyObject* args, PyObject*
     return NULL;
 }
 
+static const char PyXmlSec_KeyFromEngine__doc__[] = \
+    "from_engine(engine_and_key_id) -> xmlsec.Key\n"
+    "Loads PKI key from an engine.\n\n"
+    ":param engine_and_key_id: engine and key id, i.e. 'pkcs11;pkcs11:token=XmlsecToken;object=XmlsecKey;pin-value=password'\n"
+    ":type engine_and_key_id: :class:`str`, "
+    ":return: pointer to newly created key\n"
+    ":rtype: :class:`~xmlsec.Key`";
+static PyObject* PyXmlSec_KeyFromEngine(PyObject* self, PyObject* args, PyObject* kwargs) {
+    static char *kwlist[] = {"engine_and_key_id", NULL};
+
+    const char* engine_and_key_id = NULL;
+    PyXmlSec_Key* key = NULL;
+
+    PYXMLSEC_DEBUG("load key from engine - start");
+    if (!PyArg_ParseTupleAndKeywords(args, kwargs, "s:from_engine", kwlist, &engine_and_key_id)) {
+        goto ON_FAIL;
+    }
+
+    if ((key = PyXmlSec_NewKey1((PyTypeObject*)self)) == NULL) goto ON_FAIL;
+
+    Py_BEGIN_ALLOW_THREADS;
+    key->handle = xmlSecCryptoAppKeyLoad(engine_and_key_id, xmlSecKeyDataFormatEngine, NULL, xmlSecCryptoAppGetDefaultPwdCallback(),
+                                         (void*)engine_and_key_id);
+    Py_END_ALLOW_THREADS;
+
+    if (key->handle == NULL) {
+        PyXmlSec_SetLastError("cannot read key");
+        goto ON_FAIL;
+    }
+
+    key->is_own = 1;
+
+    PYXMLSEC_DEBUG("load key from engine - ok");
+    return (PyObject*)key;
+
+ON_FAIL:
+    PYXMLSEC_DEBUG("load key from engine - fail");
+    Py_XDECREF(key);
+    return NULL;
+}
+
 static const char PyXmlSec_KeyGenerate__doc__[] = \
     "generate(klass, size, type) -> xmlsec.Key\n"
     "Generates key of kind ``klass`` with ``size`` and ``type``.\n\n"
@@ -494,6 +535,12 @@ static PyMethodDef PyXmlSec_KeyMethods[] = {
         METH_CLASS|METH_VARARGS|METH_KEYWORDS,
         PyXmlSec_KeyFromFile__doc__
     },
+    {
+        "from_engine",
+        (PyCFunction)PyXmlSec_KeyFromEngine,
+        METH_CLASS|METH_VARARGS|METH_KEYWORDS,
+        PyXmlSec_KeyFromEngine__doc__
+    },
     {
         "generate",
         (PyCFunction)PyXmlSec_KeyGenerate,
diff --git a/src/main.c b/src/main.c
index ffcae14f..5773db3b 100644
--- a/src/main.c
+++ b/src/main.c
@@ -119,6 +119,13 @@ static PyObject* PyXmlSec_PyShutdown(PyObject* self) {
     Py_RETURN_NONE;
 }
 
+static char PyXmlSec_GetLibXmlSecVersion__doc__[] = \
+    "get_libxmlsec_version() -> tuple\n"
+    "Returns Version tuple of wrapped libxml library.";
+static PyObject* PyXmlSec_GetLibXmlSecVersion() {
+    return Py_BuildValue("(iii)", XMLSEC_VERSION_MAJOR, XMLSEC_VERSION_MINOR, XMLSEC_VERSION_SUBMINOR);
+}
+
 static char PyXmlSec_PyEnableDebugOutput__doc__[] = \
     "enable_debug_trace(enabled) -> None\n"
     "Enables or disables calling LibXML2 callback from the default errors callback.\n\n"
@@ -386,6 +393,12 @@ static PyMethodDef PyXmlSec_MainMethods[] = {
         METH_NOARGS,
         PyXmlSec_PyShutdown__doc__
     },
+    {
+        "get_libxmlsec_version",
+        (PyCFunction)PyXmlSec_GetLibXmlSecVersion,
+        METH_NOARGS,
+        PyXmlSec_GetLibXmlSecVersion__doc__
+    },
     {
         "enable_debug_trace",
         (PyCFunction)PyXmlSec_PyEnableDebugOutput,
diff --git a/src/xmlsec/__init__.pyi b/src/xmlsec/__init__.pyi
index 56356e55..6c326f56 100644
--- a/src/xmlsec/__init__.pyi
+++ b/src/xmlsec/__init__.pyi
@@ -49,6 +49,8 @@ class Key:
     @classmethod
     def from_file(cls: type[Self], file: GenericPath[AnyStr] | IO[AnyStr], format: int, password: str | None = ...) -> Self: ...
     @classmethod
+    def from_engine(cls: type[Self], engine_and_key_id: AnyStr) -> Self: ...
+    @classmethod
     def from_memory(cls: type[Self], data: AnyStr, format: int, password: str | None = ...) -> Self: ...
     @classmethod
     def generate(cls: type[Self], klass: KeyData, size: int, type: int) -> Self: ...


Test patch:
diff --git a/tests/data/sign5-out-xmlsec_1_2_36_to_37.xml b/tests/data/sign5-out-xmlsec_1_2_36_to_37.xml
new file mode 100644
index 00000000..f359b138
--- /dev/null
+++ b/tests/data/sign5-out-xmlsec_1_2_36_to_37.xml
@@ -0,0 +1,67 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!--
+XML Security Library example: Signed XML doc file (sign5 example).
+-->
+<Envelope xmlns="urn:envelope">
+  <Data>
+	Hello, World!
+  </Data>
+<Signature xmlns="http://www.w3.org/2000/09/xmldsig#">
+<SignedInfo>
+<CanonicalizationMethod Algorithm="http://www.w3.org/2001/10/xml-exc-c14n#"/>
+<SignatureMethod Algorithm="http://www.w3.org/2000/09/xmldsig#rsa-sha1"/>
+<Reference>
+<Transforms>
+<Transform Algorithm="http://www.w3.org/2000/09/xmldsig#enveloped-signature"/>
+</Transforms>
+<DigestMethod Algorithm="http://www.w3.org/2000/09/xmldsig#sha1"/>
+<DigestValue>HjY8ilZAIEM2tBbPn5mYO1ieIX4=</DigestValue>
+</Reference>
+</SignedInfo>
+<SignatureValue>SIaj/6KY3C1SmDXU2++Gm31U1xTadFp04WhBgfsJFbxrL+q7GKSKN9kfQ+UpN9+i
+D5fWmuavXEHe4Gw6RMaMEkq2URQo7F68+d5J/ajq8/l4n+xE6/reGScVwT6L4dEP
+XXVJcAi2ZnQ3O7GTNvNGCPibL9mUcyCWBFZ92Uemtc/vJFCQ7ZyKMdMfACgxOwyN
+T/9971oog241/2doudhonc0I/3mgPYWkZdX6yvr62mEjnG+oUZkhWYJ4ewZJ4hM4
+JjbFqZO+OEzDRSbw3DkmuBA/mtlx+3t13SESfEub5hqoMdVmtth/eTb64dsPdl9r
+3k1ACVX9f8aHfQQdJOmLFQ==</SignatureValue>
+<KeyInfo>
+<X509Data>
+
+
+
+<X509IssuerSerial>
+<X509IssuerName>Test Issuer</X509IssuerName>
+<X509SerialNumber>1</X509SerialNumber>
+</X509IssuerSerial>
+<X509Certificate>MIIE3zCCBEigAwIBAgIBBTANBgkqhkiG9w0BAQQFADCByzELMAkGA1UEBhMCVVMx
+EzARBgNVBAgTCkNhbGlmb3JuaWExEjAQBgNVBAcTCVN1bm55dmFsZTE9MDsGA1UE
+ChM0WE1MIFNlY3VyaXR5IExpYnJhcnkgKGh0dHA6Ly93d3cuYWxla3NleS5jb20v
+eG1sc2VjKTEZMBcGA1UECxMQUm9vdCBDZXJ0aWZpY2F0ZTEWMBQGA1UEAxMNQWxl
+a3NleSBTYW5pbjEhMB8GCSqGSIb3DQEJARYSeG1sc2VjQGFsZWtzZXkuY29tMB4X
+DTAzMDMzMTA0MDIyMloXDTEzMDMyODA0MDIyMlowgb8xCzAJBgNVBAYTAlVTMRMw
+EQYDVQQIEwpDYWxpZm9ybmlhMT0wOwYDVQQKEzRYTUwgU2VjdXJpdHkgTGlicmFy
+eSAoaHR0cDovL3d3dy5hbGVrc2V5LmNvbS94bWxzZWMpMSEwHwYDVQQLExhFeGFt
+cGxlcyBSU0EgQ2VydGlmaWNhdGUxFjAUBgNVBAMTDUFsZWtzZXkgU2FuaW4xITAf
+BgkqhkiG9w0BCQEWEnhtbHNlY0BhbGVrc2V5LmNvbTCCASIwDQYJKoZIhvcNAQEB
+BQADggEPADCCAQoCggEBAJe4/rQ/gzV4FokE7CthjL/EXwCBSkXm2c3p4jyXO0Wt
+quaNC3dxBwFPfPl94hmq3ZFZ9PHPPbp4RpYRnLZbRjlzVSOq954AXOXpSew7nD+E
+mTqQrd9+ZIbGJnLOMQh5fhMVuOW/1lYCjWAhTCcYZPv7VXD2M70vVXDVXn6ZrqTg
+qkVHE6gw1aCKncwg7OSOUclUxX8+Zi10v6N6+PPslFc5tKwAdWJhVLTQ4FKG+F53
+7FBDnNK6p4xiWryy/vPMYn4jYGvHUUk3eH4lFTCr+rSuJY8i/KNIf/IKim7g/o3w
+Ae3GM8xrof2mgO8GjK/2QDqOQhQgYRIf4/wFsQXVZcMCAwEAAaOCAVcwggFTMAkG
+A1UdEwQCMAAwLAYJYIZIAYb4QgENBB8WHU9wZW5TU0wgR2VuZXJhdGVkIENlcnRp
+ZmljYXRlMB0GA1UdDgQWBBQkhCzy1FkgYosuXIaQo6owuicanDCB+AYDVR0jBIHw
+MIHtgBS0ue+a5pcOaGUemM76VQ2JBttMfKGB0aSBzjCByzELMAkGA1UEBhMCVVMx
+EzARBgNVBAgTCkNhbGlmb3JuaWExEjAQBgNVBAcTCVN1bm55dmFsZTE9MDsGA1UE
+ChM0WE1MIFNlY3VyaXR5IExpYnJhcnkgKGh0dHA6Ly93d3cuYWxla3NleS5jb20v
+eG1sc2VjKTEZMBcGA1UECxMQUm9vdCBDZXJ0aWZpY2F0ZTEWMBQGA1UEAxMNQWxl
+a3NleSBTYW5pbjEhMB8GCSqGSIb3DQEJARYSeG1sc2VjQGFsZWtzZXkuY29tggEA
+MA0GCSqGSIb3DQEBBAUAA4GBALU/mzIxSv8vhDuomxFcplzwdlLZbvSQrfoNkMGY
+1UoS3YJrN+jZLWKSyWE3mIaPpElqXiXQGGkwD5iPQ1iJMbI7BeLvx6ZxX/f+c8Wn
+ss0uc1NxfahMaBoyG15IL4+beqO182fosaKJTrJNG3mc//ANGU9OsQM9mfBEt4oL
+NJ2D</X509Certificate>
+<X509SubjectName/>
+<X509SKI/>
+</X509Data>
+</KeyInfo>
+</Signature></Envelope>
diff --git a/tests/softhsm_setup.py b/tests/softhsm_setup.py
new file mode 100644
index 00000000..432d4b1b
--- /dev/null
+++ b/tests/softhsm_setup.py
@@ -0,0 +1,339 @@
+"""
+Testing the PKCS#11 shim layer.
+Heavily inspired by from https://github.com/IdentityPython/pyXMLSecurity by leifj
+under licence "As is", see https://github.com/IdentityPython/pyXMLSecurity/blob/master/LICENSE.txt
+"""
+
+import logging
+import os
+import shutil
+import subprocess
+import tempfile
+import traceback
+import unittest
+from typing import Dict, List, Optional, Tuple
+
+DATA_DIR = os.path.join(os.path.dirname(__file__), "data")
+
+
+def paths_for_component(component: str, default_paths: List[str]):
+    env_path = os.environ.get(component)
+    return [env_path] if env_path else default_paths
+
+
+def find_alts(component_name, alts: List[str]) -> str:
+    for a in alts:
+        if os.path.exists(a):
+            return a
+    raise unittest.SkipTest("Required component is missing: {}".format(component_name))
+
+
+def run_cmd(args, softhsm_conf=None) -> Tuple[bytes, bytes]:
+    env = {}
+    if softhsm_conf is not None:
+        env['SOFTHSM_CONF'] = softhsm_conf
+        env['SOFTHSM2_CONF'] = softhsm_conf
+    proc = subprocess.Popen(args, stdout=subprocess.PIPE, stderr=subprocess.PIPE, env=env)
+    out, err = proc.communicate()
+    if err is not None and len(err) > 0:
+        logging.error(err)
+    if out is not None and len(out) > 0:
+        logging.debug(out)
+    rv = proc.wait()
+    if rv:
+        with open(softhsm_conf) as f:
+            conf = f.read()
+        msg = '[cmd: {cmd}] [code: {code}] [stdout: {out}] [stderr: {err}] [config: {conf}]'
+        msg = msg.format(
+            cmd=" ".join(args),
+            code=rv,
+            out=out.strip(),
+            err=err.strip(),
+            conf=conf,
+        )
+        raise RuntimeError(msg)
+    return out, err
+
+
+component_default_paths: Dict[str, List[str]] = {
+    'P11_MODULE': [
+        '/usr/lib/softhsm/libsofthsm2.so',
+        '/usr/lib/x86_64-linux-gnu/softhsm/libsofthsm2.so',
+        '/usr/lib/softhsm/libsofthsm.so',
+        '/usr/lib64/softhsm/libsofthsm2.so',
+    ],
+    'P11_ENGINE': [
+        '/usr/lib/ssl/engines/libpkcs11.so',
+        '/usr/lib/engines/engine_pkcs11.so',
+        '/usr/lib/x86_64-linux-gnu/engines-1.1/pkcs11.so',
+        '/usr/lib64/engines-1.1/pkcs11.so',
+        '/usr/lib64/engines-1.1/libpkcs11.so',
+        '/usr/lib64/engines-3/pkcs11.so',
+        '/usr/lib64/engines-3/libpkcs11.so',
+        '/usr/lib/x86_64-linux-gnu/engines-3/pkcs11.so',
+        '/usr/lib/x86_64-linux-gnu/engines-3/libpkcs11.so',
+    ],
+    'PKCS11_TOOL': [
+        '/usr/bin/pkcs11-tool',
+    ],
+    'SOFTHSM': [
+        '/usr/bin/softhsm2-util',
+        '/usr/bin/softhsm',
+    ],
+    'OPENSSL': [
+        '/usr/bin/openssl',
+    ],
+}
+
+component_path: Dict[str, str] = {
+    component_name: find_alts(component_name, paths_for_component(component_name, default_paths))
+    for component_name, default_paths in component_default_paths.items()
+}
+
+softhsm_version = 1
+if component_path['SOFTHSM'].endswith('softhsm2-util'):
+    softhsm_version = 2
+
+openssl_version = subprocess.check_output([component_path['OPENSSL'], 'version'])[8:11].decode()
+
+p11_test_files: List[str] = []
+softhsm_conf: Optional[str] = None
+softhsm_db: Optional[str] = None
+
+
+def _temp_file() -> str:
+    f = tempfile.NamedTemporaryFile(delete=False)
+    p11_test_files.append(f.name)
+    return f.name
+
+
+def _temp_dir() -> str:
+    d = tempfile.mkdtemp()
+    p11_test_files.append(d)
+    return d
+
+
+@unittest.skipIf(component_path['P11_MODULE'] is None, "SoftHSM PKCS11 module not installed")
+def setup() -> None:
+    logging.debug("Creating test pkcs11 token using softhsm")
+    try:
+        global softhsm_conf
+        softhsm_conf = _temp_file()
+        logging.debug("Generating softhsm.conf")
+        with open(softhsm_conf, "w") as f:
+            if softhsm_version == 2:
+                softhsm_db = _temp_dir()
+                f.write(
+                    """
+# Generated by test
+directories.tokendir = %s
+objectstore.backend = file
+log.level = DEBUG
+"""
+                    % softhsm_db
+                )
+            else:
+                softhsm_db = _temp_file()
+                f.write(
+                    """
+# Generated by test
+0:%s
+"""
+                    % softhsm_db
+                )
+
+        logging.debug("Initializing the token")
+        out, err = run_cmd(
+            [
+                component_path['SOFTHSM'],
+                '--slot',
+                '0',
+                '--label',
+                'test',
+                '--init-token',
+                '--pin',
+                'secret1',
+                '--so-pin',
+                'secret2',
+            ],
+            softhsm_conf=softhsm_conf,
+        )
+
+        # logging.debug("Generating 1024 bit RSA key in token")
+        # run_cmd([component_path['PKCS11_TOOL'],
+        #          '--module', component_path['P11_MODULE'],
+        #          '-l',
+        #          '-k',
+        #          '--key-type', 'rsa:1024',
+        #          '--id', 'a1b2',
+        #          '--label', 'test',
+        #          '--pin', 'secret1'], softhsm_conf=softhsm_conf)
+
+        hash_priv_key = _temp_file()
+        logging.debug("Converting test private key to format for softhsm")
+        run_cmd(
+            [
+                component_path['OPENSSL'],
+                'pkcs8',
+                '-topk8',
+                '-inform',
+                'PEM',
+                '-outform',
+                'PEM',
+                '-nocrypt',
+                '-in',
+                os.path.join(DATA_DIR, 'rsakey.pem'),
+                '-out',
+                hash_priv_key,
+            ],
+            softhsm_conf=softhsm_conf,
+        )
+
+        logging.debug("Importing the test key to softhsm")
+        run_cmd(
+            [
+                component_path['SOFTHSM'],
+                '--import',
+                hash_priv_key,
+                '--token',
+                'test',
+                '--id',
+                'a1b2',
+                '--label',
+                'test',
+                '--pin',
+                'secret1',
+            ],
+            softhsm_conf=softhsm_conf,
+        )
+        run_cmd(
+            [component_path['PKCS11_TOOL'], '--module', component_path['P11_MODULE'], '-l', '--pin', 'secret1', '-O'],
+            softhsm_conf=softhsm_conf,
+        )
+        signer_cert_pem = _temp_file()
+        openssl_conf = _temp_file()
+        logging.debug("Generating OpenSSL config for version {}".format(openssl_version))
+        with open(openssl_conf, "w") as f:
+            # Might be needed with some versions of openssl, but in more recent versions dynamic_path breaks it.
+            # dynamic_path = (
+            #     "dynamic_path = %s" % component_path['P11_ENGINE']
+            #     if openssl_version.startswith(b'1.')
+            #     else ""
+            # )
+            f.write(
+                "\n".join(
+                    [
+                        "openssl_conf = openssl_def",
+                        "[openssl_def]",
+                        "engines = engine_section",
+                        "[engine_section]",
+                        "pkcs11 = pkcs11_section",
+                        "[req]",
+                        "distinguished_name = req_distinguished_name",
+                        "[req_distinguished_name]",
+                        "[pkcs11_section]",
+                        "engine_id = pkcs11",
+                        # dynamic_path,
+                        "MODULE_PATH = %s" % component_path['P11_MODULE'],
+                        "init = 0",
+                    ]
+                )
+            )
+
+        with open(openssl_conf, "r") as f:
+            logging.debug('-------- START DEBUG openssl_conf --------')
+            logging.debug(f.readlines())
+            logging.debug('-------- END DEBUG openssl_conf --------')
+        logging.debug('-------- START DEBUG paths --------')
+        logging.debug(run_cmd(['ls', '-ld', component_path['P11_ENGINE']]))
+        logging.debug(run_cmd(['ls', '-ld', component_path['P11_MODULE']]))
+        logging.debug('-------- END DEBUG paths --------')
+
+        signer_cert_der = _temp_file()
+
+        logging.debug("Generating self-signed certificate")
+        run_cmd(
+            [
+                component_path['OPENSSL'],
+                'req',
+                '-new',
+                '-x509',
+                '-subj',
+                "/CN=Test Signer",
+                '-engine',
+                'pkcs11',
+                '-config',
+                openssl_conf,
+                '-keyform',
+                'engine',
+                '-key',
+                'label_test',
+                '-passin',
+                'pass:secret1',
+                '-out',
+                signer_cert_pem,
+            ],
+            softhsm_conf=softhsm_conf,
+        )
+
+        run_cmd(
+            [
+                component_path['OPENSSL'],
+                'x509',
+                '-inform',
+                'PEM',
+                '-outform',
+                'DER',
+                '-in',
+                signer_cert_pem,
+                '-out',
+                signer_cert_der,
+            ],
+            softhsm_conf=softhsm_conf,
+        )
+
+        logging.debug("Importing certificate into token")
+
+        run_cmd(
+            [
+                component_path['PKCS11_TOOL'],
+                '--module',
+                component_path['P11_MODULE'],
+                '-l',
+                '--slot-index',
+                '0',
+                '--id',
+                'a1b2',
+                '--label',
+                'test',
+                '-y',
+                'cert',
+                '-w',
+                signer_cert_der,
+                '--pin',
+                'secret1',
+            ],
+            softhsm_conf=softhsm_conf,
+        )
+
+        # TODO: Should be teardowned in teardown:
+        os.environ['SOFTHSM_CONF'] = softhsm_conf
+        os.environ['SOFTHSM2_CONF'] = softhsm_conf
+
+    except Exception as ex:
+        print("-" * 64)
+        traceback.print_exc()
+        print("-" * 64)
+        logging.error("PKCS11 tests disabled: unable to initialize test token: %s" % ex)
+        raise ex
+
+
+def teardown() -> None:
+    global p11_test_files
+    for o in p11_test_files:
+        if os.path.exists(o):
+            if os.path.isdir(o):
+                shutil.rmtree(o)
+            else:
+                os.unlink(o)
+    p11_test_files = []
diff --git a/tests/test_ds.py b/tests/test_ds.py
index 694ad431..38f0b25c 100644
--- a/tests/test_ds.py
+++ b/tests/test_ds.py
@@ -182,7 +182,11 @@ def test_sign_case5(self):
         self.assertEqual("rsakey.pem", ctx.key.name)
 
         ctx.sign(sign)
-        self.assertEqual(self.load_xml("sign5-out.xml"), root)
+        if (1, 2, 36) <= xmlsec.get_libxmlsec_version() <= (1, 2, 37):
+            expected_xml_file = 'sign5-out-xmlsec_1_2_36_to_37.xml'
+        else:
+            expected_xml_file = 'sign5-out.xml'
+        self.assertEqual(self.load_xml(expected_xml_file), root)
 
     def test_sign_binary_bad_args(self):
         ctx = xmlsec.SignatureContext()
diff --git a/tests/test_pkcs11.py b/tests/test_pkcs11.py
new file mode 100644
index 00000000..accd29ae
--- /dev/null
+++ b/tests/test_pkcs11.py
@@ -0,0 +1,57 @@
+import xmlsec
+from tests import base
+from xmlsec import constants as consts
+
+KEY_URL = "pkcs11;pkcs11:token=test;object=test;pin-value=secret1"
+
+
+def setUpModule():
+    from tests import softhsm_setup
+
+    softhsm_setup.setup()
+
+
+def tearDownModule():
+    from tests import softhsm_setup
+
+    softhsm_setup.teardown()
+
+
+class TestKeys(base.TestMemoryLeaks):
+    def test_del_key(self):
+        ctx = xmlsec.SignatureContext(manager=xmlsec.KeysManager())
+        ctx.key = xmlsec.Key.from_engine(KEY_URL)
+        del ctx.key
+        self.assertIsNone(ctx.key)
+
+    def test_set_key(self):
+        ctx = xmlsec.SignatureContext(manager=xmlsec.KeysManager())
+        ctx.key = xmlsec.Key.from_engine(KEY_URL)
+        self.assertIsNotNone(ctx.key)
+
+    def test_sign_bad_args(self):
+        ctx = xmlsec.SignatureContext()
+        ctx.key = xmlsec.Key.from_engine(KEY_URL)
+        with self.assertRaises(TypeError):
+            ctx.sign('')
+
+    def test_sign_fail(self):
+        ctx = xmlsec.SignatureContext()
+        ctx.key = xmlsec.Key.from_engine(KEY_URL)
+        with self.assertRaisesRegex(xmlsec.Error, 'failed to sign'):
+            ctx.sign(self.load_xml('sign1-in.xml'))
+
+    def test_sign_case1(self):
+        """Should sign a pre-constructed template file using a key from a pkcs11 engine."""
+        root = self.load_xml("sign1-in.xml")
+        sign = xmlsec.tree.find_node(root, consts.NodeSignature)
+        self.assertIsNotNone(sign)
+
+        ctx = xmlsec.SignatureContext()
+        ctx.key = xmlsec.Key.from_engine(KEY_URL)
+        self.assertIsNotNone(ctx.key)
+        ctx.key.name = 'rsakey.pem'
+        self.assertEqual("rsakey.pem", ctx.key.name)
+
+        ctx.sign(sign)
+        self.assertEqual(self.load_xml("sign1-out.xml"), root)