Repository: privacyidea/privacyidea

Issue:
'NoneType' object has no attribute 'encryptionkey' when importing pskc
<!--
Thank you for filing a bug within privacyIDEA. The more information you provide, the easier it is to identify the problem!

Issues are not meant for asking questions! You're welcome to ask questions in our community forum: https://community.privacyidea.org

privacyIDEA is licensed under the AGPL and comes without warranty. If you require reliable, professional help, please consider getting either
- consultancy 
  (https://netknights.it/en/leistungen/one-time-services/)
- or the privacyIDEA Enterprise Edition 
  (https://netknights.it/en/leistungen/service-level-agreements/)
-->

### Top-level intent

What did you try to achieve?

Load a pskc file into PrivacyIDEA

### Steps to reproduce

1. Clean install
2. Log in, make realm from passwd file 
3. Import Token -> PSKC -> select file

### Expected outcome

What do you think, how the system should have worked?

It should have imported the token

### Actual outcome

What did actually happen?

Got an error

### Configuration

* **privacyIDEA version**: 3.0
* **Installation method**: (from Ubuntu packages, github, PyPI, ...) Ubuntu
* **Python version**: I have 2.7 and 3.6 installed
* **Operating system**: Ubuntu 18.04
* **Webserver**: apache2 
* **Token database**: (MySQL, PostgreSQL, ...) mysql

### Log file

[2019-11-05 13:11:34,024][754][140683384121088][DEBUG][privacyidea.api.before_after:84] Begin handling of request u'/token/load/filename?'
[2019-11-05 13:11:34,025][754][140683384121088][DEBUG][privacyidea.api.lib.utils:219] Can not get param: No JSON object could be decoded
[2019-11-05 13:11:34,025][754][140683384121088][DEBUG][privacyidea.lib.user:185] Entering get_user_from_param with arguments ({'tokenrealms': u'defrealm', 'type': u'pskc'},) and keywords {}
[2019-11-05 13:11:34,025][754][140683384121088][DEBUG][privacyidea.lib.user:185] Entering User with arguments () and keywords {'login': '', 'realm': '', 'resolver': None}
[2019-11-05 13:11:34,025][754][140683384121088][DEBUG][privacyidea.lib.user:197] Exiting User with result <empty user>
[2019-11-05 13:11:34,025][754][140683384121088][DEBUG][privacyidea.lib.user:197] Exiting get_user_from_param with result <empty user>
[2019-11-05 13:11:34,025][754][140683384121088][DEBUG][privacyidea.lib.audit:188] Entering getAudit with arguments HIDDEN and keywords HIDDEN
[2019-11-05 13:11:34,026][754][140683384121088][DEBUG][privacyidea.lib.utils:1120] klass: <class 'privacyidea.lib.auditmodules.sqlaudit.Audit'>
[2019-11-05 13:11:34,026][754][140683384121088][DEBUG][privacyidea.lib.auditmodules.base:185] Entering read_keys with arguments (<privacyidea.lib.auditmodules.sqlaudit.Audit object at 0x7ff357334a90>, '/etc/privacyidea/public.pem', '/etc/privacyidea/private.pem') and keywords {}
[2019-11-05 13:11:34,026][754][140683384121088][DEBUG][privacyidea.lib.auditmodules.base:197] Exiting read_keys with result None
[2019-11-05 13:11:34,026][754][140683384121088][DEBUG][privacyidea.lib.audit:197] Exiting getAudit with result <privacyidea.lib.auditmodules.sqlaudit.Audit object at 0x7ff357334a90>
[2019-11-05 13:11:34,029][754][140683384121088][DEBUG][privacyidea.lib.config:185] Entering get_from_config with arguments ('OverrideAuthorizationClient',) and keywords {}
[2019-11-05 13:11:34,029][754][140683384121088][DEBUG][privacyidea.lib.config:315] Cloning request-local config from shared config object
[2019-11-05 13:11:34,031][754][140683384121088][DEBUG][privacyidea.lib.config:197] Exiting get_from_config with result None
[2019-11-05 13:11:34,031][754][140683384121088][DEBUG][privacyidea.lib.auditmodules.base:185] Entering log with arguments (<privacyidea.lib.auditmodules.sqlaudit.Audit object at 0x7ff357334a90>, {'privacyidea_server': '192.168.2.230', 'token_type': None, 'user': None, 'resolver': None, 'serial': None, 'info': '', 'realm': None, 'success': False, 'client_user_agent': 'firefox', 'client': '192.168.2.147', 'action_detail': '', 'action': 'POST /token/load/<filename>'}) and keywords {}
[2019-11-05 13:11:34,031][754][140683384121088][DEBUG][privacyidea.lib.auditmodules.base:197] Exiting log with result None
[2019-11-05 13:11:34,032][754][140683384121088][DEBUG][privacyidea.lib.auditmodules.base:185] Entering log with arguments (<privacyidea.lib.auditmodules.sqlaudit.Audit object at 0x7ff357334a90>, {'administrator': u'admin'}) and keywords {}
[2019-11-05 13:11:34,032][754][140683384121088][DEBUG][privacyidea.lib.auditmodules.base:197] Exiting log with result None
[2019-11-05 13:11:34,033][754][140683384121088][DEBUG][privacyidea.api.token:185] Entering loadtokens_api with arguments () and keywords {'filename': u'filename'}
[2019-11-05 13:11:34,033][754][140683384121088][DEBUG][privacyidea.lib.policy:185] Entering list_policies with arguments (<privacyidea.lib.policy.PolicyClass object at 0x7ff3572c2b10>,) and keywords {'realm': None, 'name': None, 'sort_by_priority': True, 'action': 'importtokens', 'client': '192.168.2.147', 'user': u'admin', 'resolver': None, 'active': True, 'scope': 'admin', 'adminrealm': u''}
[2019-11-05 13:11:34,033][754][140683384121088][DEBUG][privacyidea.lib.policy:515] Policies after matching active: [][2019-11-05 13:11:34,033][754][140683384121088][DEBUG][privacyidea.lib.policy:515] Policies after matching scope: []
[2019-11-05 13:11:34,033][754][140683384121088][DEBUG][privacyidea.lib.policy:539] Policies after matching action: [][2019-11-05 13:11:34,033][754][140683384121088][DEBUG][privacyidea.lib.policy:539] Policies after matching user: []
[2019-11-05 13:11:34,034][754][140683384121088][DEBUG][privacyidea.lib.policy:539] Policies after matching adminrealm: []
[2019-11-05 13:11:34,034][754][140683384121088][DEBUG][privacyidea.lib.policy:605] Policies after matching client
[2019-11-05 13:11:34,034][754][140683384121088][DEBUG][privacyidea.lib.policy:197] Exiting list_policies with result []
[2019-11-05 13:11:34,034][754][140683384121088][DEBUG][privacyidea.lib.policy:665] Policies after matching time: []
[2019-11-05 13:11:34,034][754][140683384121088][DEBUG][privacyidea.lib.policy:670] Policies after matching conditions[2019-11-05 13:11:34,034][754][140683384121088][DEBUG][privacyidea.lib.policy:185] Entering list_policies with arguments (<privacyidea.lib.policy.PolicyClass object at 0x7ff3572c2b10>,) and keywords {'active': True, 'scope': 'admin'}
[2019-11-05 13:11:34,034][754][140683384121088][DEBUG][privacyidea.lib.policy:515] Policies after matching active: [][2019-11-05 13:11:34,034][754][140683384121088][DEBUG][privacyidea.lib.policy:515] Policies after matching scope: []
[2019-11-05 13:11:34,034][754][140683384121088][DEBUG][privacyidea.lib.policy:197] Exiting list_policies with result []
[2019-11-05 13:11:34,035][754][140683384121088][DEBUG][privacyidea.api.token:885] Werkzeug File storage file: <FileStorage: u'gemalto.pskc' ('application/octet-stream')>
[2019-11-05 13:11:34,035][754][140683384121088][DEBUG][privacyidea.lib.importotp:185] Entering parsePSKCdata with arguments (u'<pskc:KeyPackage>\r\n    <pskc:DeviceInfo>\r\n      <pskc:Manufacturer>iana.GEMALTO</pskc:Manufacturer>\r\n      <pskc:SerialNo><redacted></pskc:SerialNo>\r\n      <pskc:Model>LAVA</pskc:Model>\r\n      <pskc:IssueNo>1</pskc:IssueNo>\r\n    </pskc:DeviceInfo>\r\n    <pskc:Key Id="<redacted>" Algorithm="urn:ietf:params:xml:ns:keyprov:pskc:totp">\r\n      <pskc:Issuer>Gemalto</pskc:Issuer>\r\n      <pskc:AlgorithmParameters>\r\n        <pskc:ResponseFormat Encoding="DECIMAL" Length="7" />\r\n      </pskc:AlgorithmParameters>\r\n      <pskc:Data>\r\n        <pskc:Secret>\r\n          <pskc:EncryptedValue>\r\n            <xenc:EncryptionMethod Algorithm="http://www.w3.org/2001/04/xmlenc#aes128-cbc" />\r\n            <xenc:CipherData>\r\n              <xenc:CipherValue><redacted></xenc:CipherValue>\r\n            </xenc:CipherData>\r\n          </pskc:EncryptedValue>\r\n          <pskc:ValueMAC><redacted></pskc:ValueMAC>\r\n        </pskc:Secret>\r\n        <pskc:Time>\r\n          <pskc:PlainValue>0</pskc:PlainValue>\r\n        </pskc:Time>\r\n        <pskc:TimeInterval>\r\n          <pskc:PlainValue>30</pskc:PlainValue>\r\n        </pskc:TimeInterval>\r\n      </pskc:Data>\r\n      <pskc:Policy>\r\n        <pskc:KeyUsage>OTP</pskc:KeyUsage>\r\n      </pskc:Policy>\r\n    </pskc:Key>\r\n  </pskc:KeyPackage>',) and keywords {'password': None, 'preshared_key_hex': None}
[2019-11-05 13:11:34,139][754][140683384121088][DEBUG][privacyidea.lib.auditmodules.base:185] Entering log with arguments (<privacyidea.lib.auditmodules.sqlaudit.Audit object at 0x7ff357334a90>, {'info': u"'NoneType' object has no attribute 'encryptionkey'"}) and keywords {}
[2019-11-05 13:11:34,139][754][140683384121088][DEBUG][privacyidea.lib.auditmodules.base:197] Exiting log with result None
[2019-11-05 13:11:34,198][754][140683384121088][DEBUG][privacyidea.api.lib.utils:219] Can not get param: No JSON object could be decoded
[2019-11-05 13:11:34,200][754][140683384121088][DEBUG][privacyidea.api.lib.utils:219] Can not get param: No JSON object could be decoded

**Set PI_LOGLEVEL = logging.DEBUG in pi.cfg and take a look at the privacyidea.log!**
**If appropriate, attach the log file or paste relevant portions.**

Note: Sensitive data has been replaced with <redacted>


Discussion:

Resolving patch:
diff --git a/privacyidea/api/token.py b/privacyidea/api/token.py
--- a/privacyidea/api/token.py
+++ b/privacyidea/api/token.py
@@ -968,7 +968,8 @@ def loadtokens_api(filename=None):
     g.audit_object.log({'info': u"{0!s}, {1!s} (imported: {2:d})".format(file_type,
                                                            token_file,
                                                            len(TOKENS)),
-                        'serial': ', '.join(TOKENS)})
+                        'serial': ', '.join(TOKENS),
+                        'success': True})
     # logTokenNum()
 
     return send_result(len(TOKENS))
diff --git a/privacyidea/lib/importotp.py b/privacyidea/lib/importotp.py
--- a/privacyidea/lib/importotp.py
+++ b/privacyidea/lib/importotp.py
@@ -341,7 +341,11 @@ def parseSafeNetXML(xml):
     """
 
     TOKENS = {}
-    elem_tokencontainer = etree.fromstring(xml)
+    try:
+        elem_tokencontainer = etree.fromstring(xml)
+    except etree.ParseError as e:
+        log.debug(traceback.format_exc())
+        raise ImportException('Could not parse XML data: {0!s}'.format(e))
 
     if getTagName(elem_tokencontainer) != "Tokens":
         raise ImportException("No toplevel element Tokens")
@@ -461,9 +465,11 @@ def parsePSKCdata(xml_data,
     """
 
     tokens = {}
-    #xml = BeautifulSoup(xml_data, "lxml")
     xml = strip_prefix_from_soup(BeautifulSoup(xml_data, "lxml"))
 
+    if not xml.keycontainer:
+        raise ImportException("No KeyContainer found in PSKC data. Could not "
+                              "import any tokens.")
     if xml.keycontainer.encryptionkey and \
             xml.keycontainer.encryptionkey.derivedkey:
         # If we have a password we also need a tag EncryptionKey in the


Test patch:
diff --git a/tests/test_api_token.py b/tests/test_api_token.py
--- a/tests/test_api_token.py
+++ b/tests/test_api_token.py
@@ -18,7 +18,6 @@
 from dateutil.tz import tzlocal
 from privacyidea.lib import _
 
-PWFILE = "tests/testdata/passwords"
 IMPORTFILE = "tests/testdata/import.oath"
 IMPORTFILE_GPG = "tests/testdata/import.oath.asc"
 IMPORTFILE2 = "tests/testdata/empty.oath"
@@ -28,44 +27,6 @@
 YUBICOFILE = "tests/testdata/yubico-oath.csv"
 OTPKEY = "3132333435363738393031323334353637383930"
 OTPKEY2 = "010fe88d31948c0c2e3258a4b0f7b11956a258ef"
-OTPVALUES2 = ["551536", "703671", "316522", "413789"]
-CERT = """-----BEGIN CERTIFICATE-----
-MIIGXDCCBUSgAwIBAgITYwAAAA27DqXl0fVdOAAAAAAADTANBgkqhkiG9w0BAQsF
-ADBCMRMwEQYKCZImiZPyLGQBGRYDb3JnMRkwFwYKCZImiZPyLGQBGRYJYXV0aC10
-ZXN0MRAwDgYDVQQDEwdDQTJGMDAxMB4XDTE1MDIxMTE2NDE1M1oXDTE2MDIxMTE2
-NDE1M1owgYExEzARBgoJkiaJk/IsZAEZFgNvcmcxGTAXBgoJkiaJk/IsZAEZFglh
-dXRoLXRlc3QxDjAMBgNVBAMTBVVzZXJzMRowGAYDVQQDExFDb3JuZWxpdXMgS29l
-bGJlbDEjMCEGCSqGSIb3DQEJARYUY29ybmVsaXVzQGJhbGZvby5uZXQwggEiMA0G
-CSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQCN5xYqSoKhxKgywdWOjZTgOobPN5lN
-DbSKQktdiG7asH0/Bzg8DIyd+k6wj5yncNhHKBhDJC/cAz3YAYY+KJj/tECLyt5V
-AqZLuf3sTA/Ak/neHzXwrlo9PB67JxY4tgJcaR0Cml5oSx4ofRowOCrXv60Asfkl
-+3lMRaNyEpQiSVdqIzGZAM1FIy0chwknMB8PfQhlC3v60rGiWoG65Rl5zuGl9lJC
-nR990FGSIUW2GLCtI57QCCdBVHIBL+M0WNbdonk9qYSHm8ArFeoftsw2UxHQazM9
-KftS7osJnQWOeNw+iIQIgZxLlyC9CBeKBCj3gIwLMEZRz6y951A9nngbAgMBAAGj
-ggMJMIIDBTAOBgNVHQ8BAf8EBAMCBLAwHQYDVR0OBBYEFGFYluib3gs1BQQNB25A
-FyEvQuoxMB8GA1UdIwQYMBaAFO9tVOusjflOf/y1lJuQ0YZej3vuMIHHBgNVHR8E
-gb8wgbwwgbmggbaggbOGgbBsZGFwOi8vL0NOPUNBMkYwMDEsQ049Z2FuZGFsZixD
-Tj1DRFAsQ049UHVibGljJTIwS2V5JTIwU2VydmljZXMsQ049U2VydmljZXMsQ049
-Q29uZmlndXJhdGlvbixEQz1hdXRoLXRlc3QsREM9b3JnP2NlcnRpZmljYXRlUmV2
-b2NhdGlvbkxpc3Q/YmFzZT9vYmplY3RDbGFzcz1jUkxEaXN0cmlidXRpb25Qb2lu
-dDCBuwYIKwYBBQUHAQEEga4wgaswgagGCCsGAQUFBzAChoGbbGRhcDovLy9DTj1D
-QTJGMDAxLENOPUFJQSxDTj1QdWJsaWMlMjBLZXklMjBTZXJ2aWNlcyxDTj1TZXJ2
-aWNlcyxDTj1Db25maWd1cmF0aW9uLERDPWF1dGgtdGVzdCxEQz1vcmc/Y0FDZXJ0
-aWZpY2F0ZT9iYXNlP29iamVjdENsYXNzPWNlcnRpZmljYXRpb25BdXRob3JpdHkw
-PQYJKwYBBAGCNxUHBDAwLgYmKwYBBAGCNxUIheyJB4SuoT6EjYcBh+WGHoXd8y83
-g7DpBYPZgFwCAWQCAQgwKQYDVR0lBCIwIAYKKwYBBAGCNxQCAgYIKwYBBQUHAwIG
-CCsGAQUFBwMEMDUGCSsGAQQBgjcVCgQoMCYwDAYKKwYBBAGCNxQCAjAKBggrBgEF
-BQcDAjAKBggrBgEFBQcDBDBEBgNVHREEPTA7oCMGCisGAQQBgjcUAgOgFQwTY29y
-bnlAYXV0aC10ZXN0Lm9yZ4EUY29ybmVsaXVzQGJhbGZvby5uZXQwRAYJKoZIhvcN
-AQkPBDcwNTAOBggqhkiG9w0DAgICAIAwDgYIKoZIhvcNAwQCAgCAMAcGBSsOAwIH
-MAoGCCqGSIb3DQMHMA0GCSqGSIb3DQEBCwUAA4IBAQCVI9ULYQgLxOcDWAlWPE4g
-ZRcbg65oCNdB0MBzTFhQZC/YFlSTNAGU2gUhnW+LoQ4N4sVnwxPbCRpsiA0ImqFU
-hh/qcIV4JYthUGYdYkGjsc1YQjdLpYsg0GRUXTQHYjMQo6gvg1z/iMhzCCU8DbjT
-DkTm/0JYVCt+vpvpigX/XWLWeHLHzPHFYAdBVAYgnwbTV4hgNIO98YRiMWsXOAIR
-S/IreZ58alclwJJRIGTuOTKSCd+uE7QMALztDty7cjtpMANGrz1k/uUWg9T+UgQs
-czZ68tF258iaWLPbsdRWqO160iy7eDSKWFFMR4HnfLHX/UPRSpBNGSHmvT1hbkUr
------END CERTIFICATE-----"""
-
 CAKEY = "cakey.pem"
 CACERT = "cacert.pem"
 OPENSSLCNF = "openssl.cnf"
@@ -883,6 +844,9 @@ def test_11_load_tokens(self):
             result = res.json.get("result")
             value = result.get("value")
             self.assertTrue(value == 3, result)
+        # check for a successful audit entry
+        entry = self.find_most_recent_audit_entry(action='*/token/load/*')
+        self.assertEqual(entry['success'], 1, entry)
 
         # Load GPG encrypted OATH CSV
         with self.app.test_request_context('/token/load/import.oath.asc',
@@ -927,6 +891,9 @@ def test_11_load_tokens(self):
                                             headers={'Authorization': self.at}):
             res = self.app.full_dispatch_request()
             self.assertTrue(res.status_code == 400, res)
+        # check for a failed audit entry
+        entry = self.find_most_recent_audit_entry(action='*/token/load/*')
+        self.assertEqual(entry['success'], 0, entry)
 
         # Try to load unknown file type
         with self.app.test_request_context('/token/load/import.oath',
diff --git a/tests/test_lib_importotp.py b/tests/test_lib_importotp.py
--- a/tests/test_lib_importotp.py
+++ b/tests/test_lib_importotp.py
@@ -540,6 +540,7 @@ def test_00_import_oath_user(self):
         self.assertEqual(tokens.get("tok3").get("user").get("resolver"), "resolver3")
 
     def test_01_import_aladdin_xml(self):
+        self.assertRaises(ImportException, parseSafeNetXML, 'no xml')
         tokens = parseSafeNetXML(ALADDINXML)
         self.assertTrue(len(tokens) == 2)
         self.assertTrue("00040008CFA52" in tokens, tokens)
@@ -554,6 +555,8 @@ def test_02_import_yubikey(self):
         self.assertTrue("UBAM00508326_1" in tokens, tokens)
 
     def test_03_import_pskc(self):
+        self.assertRaises(ImportException, parsePSKCdata, 'not xml')
+
         tokens = parsePSKCdata(XML_PSKC)
         self.assertEqual(len(tokens), 7)
         self.assertEqual(tokens["1000133508267"].get("type"), "hotp")
diff --git a/tests/testdata/aladdin.xml b/tests/testdata/aladdin.xml
deleted file mode 100644
--- a/tests/testdata/aladdin.xml
+++ /dev/null
@@ -1,67 +0,0 @@
-        <Tokens>
-        <Token serial="00040008CFA5">
-        <CaseModel>5</CaseModel>
-        <Model>101</Model>
-        <ProductionDate>02/19/2009</ProductionDate>
-        <ProductName>Safeword Alpine</ProductName>
-        <Applications>
-        <Application ConnectorID="{ab1397d2-ddb6-4705-b66e-9f83f322deb9}">
-        <Seed>123412354</Seed>
-        <MovingFactor>1</MovingFactor>
-        </Application>
-        </Applications>
-        </Token>
-
-        <Token serial="00040008CFA52">
-        <CaseModel>5</CaseModel>
-        <Model>101</Model>
-        <ProductionDate>02/19/2009</ProductionDate>
-        <ProductName>Safeword Alpine</ProductName>
-        <Applications>
-        <Application ConnectorID="{ab1397d2-ddb6-4705-b66e-9f83f322deb9}">
-        <Seed>123456</Seed>
-        <MovingFactor>1</MovingFactor>
-        </Application>
-        </Applications>
-        </Token>
-
-        <Token info="token without serial">
-        <CaseModel>5</CaseModel>
-        <Model>101</Model>
-        <ProductionDate>02/19/2009</ProductionDate>
-        <ProductName>Safeword Alpine</ProductName>
-        <Applications>
-        <Application ConnectorID="{ab1397d2-ddb6-4705-b66e-9f83f322deb9}">
-        <Seed>123456</Seed>
-        <MovingFactor>1</MovingFactor>
-        </Application>
-        </Applications>
-        </Token>
-
-        <Token serial="token_without_seed">
-        <CaseModel>5</CaseModel>
-        <Model>101</Model>
-        <ProductionDate>02/19/2009</ProductionDate>
-        <ProductName>Safeword Alpine</ProductName>
-        <Applications>
-        <Application ConnectorID="{ab1397d2-ddb6-4705-b66e-9f83f322deb9}">
-        <MovingFactor>1</MovingFactor>
-        </Application>
-        </Applications>
-        </Token>
-
-        <Token serial="sha256">
-        <CaseModel>5</CaseModel>
-        <Model>101</Model>
-        <ProductionDate>02/19/2009</ProductionDate>
-        <ProductName>Safeword Alpine</ProductName>
-        <Applications>
-        <Application ConnectorID="{ab1397d2-ddb6-4705-b66e-9f83f322deb9}">
-        <seed>1234567890123456789012345678901234567890123456789012345678901234
-        </seed>
-        <MovingFactor>1</MovingFactor>
-        </Application>
-        </Applications>
-        </Token>
-
-        </Tokens>