Repository: StackStorm/st2

Issue:
st2api encrypt of £ (and other utf-8 chars) fails with python stacktrace 
##### SUMMARY

Provide a quick summary of your bug or feature request.

##### ISSUE TYPE
Pick one below and delete the rest:
 - Bug Report

##### STACKSTORM VERSION
Paste the output of `st2 --version`:
root@stackstorm-ha-st2client-68fdcb57f-xnxkv:/opt/stackstorm# st2 --version
st2 3.0dev (9577ff944), on Python 3.6.7

##### OS / ENVIRONMENT / INSTALL METHOD
kubernetes using ubuntu 18.04 using helm

##### STEPS TO REPRODUCE

root@stackstorm-ha-st2client-68fdcb57f-xnxkv:/opt/stackstorm# st2 key set --scope=system --encrypt test '£'
ERROR: 400 Client Error: Bad Request
MESSAGE: The length of the provided data is not a multiple of the block length. for url: http://stackstorm-ha-st2api:9101/keys/test

During handling of the above exception, another exception occurred:
#### error log from the st2api container
2019-01-23 16:06:14,301 ERROR [-] Failed to call controller function "put" for operation "st2api.controllers.v1.keyvalue:key_value_pair_controller.put": The length of the provided data is not a multiple of the block length.
Traceback (most recent call last):
  File "/opt/stackstorm/st2/lib/python3.6/site-packages/st2api/controllers/v1/keyvalue.py", line 301, in put
    kvp_db = KeyValuePairAPI.to_model(kvp)
  File "/opt/stackstorm/st2/lib/python3.6/site-packages/st2common/models/api/keyvalue.py", line 175, in to_model
    value = symmetric_encrypt(KeyValuePairAPI.crypto_key, value)
  File "/opt/stackstorm/st2/lib/python3.6/site-packages/st2common/util/crypto.py", line 186, in symmetric_encrypt
    return cryptography_symmetric_encrypt(encrypt_key=encrypt_key, plaintext=plaintext)
  File "/opt/stackstorm/st2/lib/python3.6/site-packages/st2common/util/crypto.py", line 237, in cryptography_symmetric_encrypt
    ciphertext_bytes = encryptor.update(data) + encryptor.finalize()
  File "/opt/stackstorm/st2/lib/python3.6/site-packages/cryptography/hazmat/primitives/ciphers/base.py", line 159, in finalize
    data = self._ctx.finalize()
  File "/opt/stackstorm/st2/lib/python3.6/site-packages/cryptography/hazmat/backends/openssl/ciphers.py", line 173, in finalize
    "The length of the provided data is not a multiple of "
ValueError: The length of the provided data is not a multiple of the block length.

##### EXPECTED RESULTS
we should be able to encrypt the £ sign

##### ACTUAL RESULTS
http 400 and a python stacktrace on the api



Discussion:
make that utf-8 special characters
Update: the tests failed for that pull request, I added a comment describing the issue but don't have a solution yet.
Thanks for reporting this issue and sorry for the delay.

We will have a look as soon as we get a chance.

Resolving patch:
diff --git a/st2common/st2common/util/crypto.py b/st2common/st2common/util/crypto.py
--- a/st2common/st2common/util/crypto.py
+++ b/st2common/st2common/util/crypto.py
@@ -216,8 +216,14 @@ def cryptography_symmetric_encrypt(encrypt_key, plaintext):
     assert isinstance(aes_key_bytes, six.binary_type)
     assert isinstance(hmac_key_bytes, six.binary_type)
 
+    if isinstance(plaintext, (six.text_type, six.string_types)):
+        # Convert data to bytes
+        data = plaintext.encode('utf-8')
+    else:
+        data = plaintext
+
     # Pad data
-    data = pkcs5_pad(plaintext)
+    data = pkcs5_pad(data)
 
     # Generate IV
     iv_bytes = os.urandom(KEYCZAR_AES_BLOCK_SIZE)
@@ -230,10 +236,6 @@ def cryptography_symmetric_encrypt(encrypt_key, plaintext):
     # bytes) so we simply add 5 0's
     header_bytes = b'00000'
 
-    if isinstance(data, (six.text_type, six.string_types)):
-        # Convert data to bytes
-        data = data.encode('utf-8')
-
     ciphertext_bytes = encryptor.update(data) + encryptor.finalize()
     msg_bytes = header_bytes + iv_bytes + ciphertext_bytes
 
@@ -368,7 +370,7 @@ def pkcs5_pad(data):
     Pad data using PKCS5
     """
     pad = KEYCZAR_AES_BLOCK_SIZE - len(data) % KEYCZAR_AES_BLOCK_SIZE
-    data = data + pad * chr(pad)
+    data = data + pad * chr(pad).encode('utf-8')
     return data
 
 


Test patch:
diff --git a/st2common/tests/unit/test_crypto_utils.py b/st2common/tests/unit/test_crypto_utils.py
--- a/st2common/tests/unit/test_crypto_utils.py
+++ b/st2common/tests/unit/test_crypto_utils.py
@@ -1,3 +1,4 @@
+# -*- coding: utf-8 -*-
 # Licensed to the StackStorm, Inc ('StackStorm') under one or more
 # contributor license agreements.  See the NOTICE file distributed with
 # this work for additional information regarding copyright ownership.
@@ -53,6 +54,31 @@ def setUpClass(cls):
         super(CryptoUtilsTestCase, cls).setUpClass()
         CryptoUtilsTestCase.test_crypto_key = AESKey.generate()
 
+    def test_symmetric_encrypt_decrypt_short_string_needs_to_be_padded(self):
+        original = u'a'
+        crypto = symmetric_encrypt(CryptoUtilsTestCase.test_crypto_key, original)
+        plain = symmetric_decrypt(CryptoUtilsTestCase.test_crypto_key, crypto)
+        self.assertEqual(plain, original)
+
+    def test_symmetric_encrypt_decrypt_utf8_character(self):
+        values = [
+            u'£',
+            u'£££',
+            u'££££££',
+            u'č š hello đ č p ž Ž',
+            u'hello 💩',
+            u'💩💩💩💩💩'
+            u'💩💩💩',
+            u'💩😁'
+        ]
+
+        for index, original in enumerate(values):
+            crypto = symmetric_encrypt(CryptoUtilsTestCase.test_crypto_key, original)
+            plain = symmetric_decrypt(CryptoUtilsTestCase.test_crypto_key, crypto)
+            self.assertEqual(plain, original)
+
+        self.assertEqual(index, (len(values) - 1))
+
     def test_symmetric_encrypt_decrypt(self):
         original = 'secret'
         crypto = symmetric_encrypt(CryptoUtilsTestCase.test_crypto_key, original)