Cryptanalysis of Some AES-based Cryptographic Primitives

Current information security systems rely heavily on symmetric key cryptographic primitives as one of their basic building blocks. In order to boost the efficiency of the security systems, designers of the underlying primitives often tend to avoid the use of provably secure designs. In fact, they adopt ad hoc designs with claimed security assumptions in the hope that they resist known cryptanalytic attacks. Accordingly, the security evaluation of such primitives continually remains an open field. In this thesis, we analyze the security of two cryptographic hash functions and one block cipher. We primarily focus on the recent AES-based designs used in the new Russian Federation cryptographic hashing and encryption suite GOST because the majority of our work was carried out during the open research competition run by the Russian standardization body TC26 for the analysis of their new cryptographic hash function Streebog. Although, there exist security proofs for the resistance of AES- based primitives against standard differential and linear attacks, other cryptanalytic techniques such as integral, rebound, and meet-in-the-middle attacks have proven to be effective. The results presented in this thesis can be summarized as follows: Initially, we analyze various security aspects of the Russian cryptographic hash function GOST R 34.11-2012, also known as Streebog or Stribog. In particular, our work investigates five security aspects of Streebog. Firstly, we present a collision analysis of the compression function and its in- ternal cipher in the form of a series of modified rebound attacks. Secondly, we propose an integral distinguisher for the 7- and 8-round compression function. Thirdly, we investigate the one wayness of Streebog with respect to two approaches of the meet-in-the-middle attack, where we present a preimage analysis of the compression function and combine the results with a multicollision attack to generate a preimage of the hash function output. Fourthly, we investigate Streebog in the context of malicious hashing and by utilizing a carefully tailored differential path, we present a backdoored version of the hash function where collisions can be generated with practical complexity. Lastly, we propose a fault analysis attack which retrieves the inputs of the compression function and utilize it to recover the secret key when Streebog is used in the keyed simple prefix and secret-IV MACs, HMAC, or NMAC. All the presented results are on reduced round variants of the function except for our analysis of the malicious version of Streebog and our fault analysis attack where both attacks cover the full round hash function. Next, we examine the preimage resistance of the AES-based Maelstrom-0 hash function which is designed to be a lightweight alternative to the ISO standardized hash function Whirlpool. One of the distinguishing features of the Maelstrom-0 design is the proposal of a new chaining construction called 3CM which is based on the 3C/3C+ family. In our analysis, we employ a 4-stage approach that uses a modified technique to defeat the 3CM chaining construction and generates preimages of the 6-round reduced Maelstrom-0 hash function. Finally, we provide a key recovery attack on the new Russian encryption standard GOST R 34.12- 2015, also known as Kuznyechik. Although Kuznyechik adopts an AES-based design, it exhibits a faster diffusion rate as it employs an optimal diffusion transformation. In our analysis, we propose a meet-in-the-middle attack using the idea of efficient differential enumeration where we construct a three round distinguisher and consequently are able to recover 16-bytes of the master key of the reduced 5-round cipher. We also present partial sequence matching, by which we generate, store, and match parts of the compared parameters while maintaining negligible probability of matching error, thus the overall online time complexity of the attack is reduced.