Breaking the Enigma Code in Python with MCMC (Marvel themed)
By: Jack J
Tags: Machine Learning, Data Science, Enigma, Cryptography, Markov Chains

An enigma machine. Source

Knowledge is Power. This rule can be applied to almost any domain, yet it is arguably never more relevant than during times of war. The ability to predict your enemies next move is equally difficult as it is rewarding, a task which is made considerably easier if one could read their plans ahead of time.

This is something all sides knew during World War II, and Nazi Germany put great effort into ensuring the privacy and security of their communication. A primary tool for this was the “Engima”, a machine which allowed their correspondents to be safely encrypted and decrypted through a simple process of letter switching (all ‘a’s become ‘z’s, for instance).

Although the encryption process was simple (by today’s standards, at least), it was a tremendous task for the Allies to find the correct keys to decrypt the intercepted messages. This was compounded by the fact that these keys would switch daily, making all efforts up to that point futile. Of course, as you may know, the challenge was met by Alan Turing and his team at Bletchley Park which led to a breakthrough in computing and became a foundation for “Artificial Intelligence”.

However, this article is not about Turing or his methods (albeit they share some qualities). Nor will this be about decrypting Nazi communications. Instead, we will take a geekier tact and look at a (fictional) cryptic message intercepted by agents of S.H.I.E.L.D from the genocidal warlord of Titan, Thanos (the central villain that concluded the initial 22-film Marvel cinematic universe). You will play the part of a S.H.I.E.L.D analyst, who only through the use of Python and his knowledge of Markov Chain Monte Carlo (MCMC) will attempt to decrypt the code and save the Universe.

If you are interested in the actual Enigma machine and how it was decrypted, I recommend starting with this Numberphile video here: https://www.youtube.com/watch?v=G2_Q9FoD-oQ

"huvw !kzcq7nv3 qv3!hqv ,evwrhqv3rvq9qwi3qvhuv2cq,3vmk,7v,7zvbikb!kvhuvzqe3!7ufv3 qvmk,7nv,evuriv57r1nv!ev3rvieqv3 qvwreh!wvq73!3!qev57r17v,ev3 qv!7b!7!3uve3r7qev3rv1!mqv ,kbvrbv3 qvi7!;qceqvri3vrbvq9!e3q7wqverv3 ,3v3 qvr3 qcv ,kbvh,uv3 c!;qfvricv2cq,3qe3vw ,kkq72qv ,evtqq7vb!7z!72v3 qve3r7qev!7v3 !ev;,e3vi7!;qceqnvti3v7r1v3 quv,cqv1!3 !7vricv2c,emfv3 qvem,wqve3r7qnv57r1ev,kerv,ev3 qv3qeeqc,w3nvcqe!zqevr7v,e2,czfv,b3qcvkr5!evb,!kicqvr7vq,c3 nv3 rcv ,ev3,5q7v!3v,7zvmi3v!3v!7vrz!7ev;,ik3fv3 qvmr1qcve3r7qv!evr7vhrc,2fv!v ,;qvrczqcqzvcr7,7v3 qv,wwieqcv3rvcq3c!q;qv!3vbrcvhqnv!7vcq3ic7v!ve ,kkvzqe3cruv9,7z,cfv3 qvh!7zve3r7qv!evr7vq,c3 nv!vmi3v!3v!7v3 qve3,bbv!v2,;qv3rvkr5!nvti3v7r1v!v ,;qv q,czv3 ,3v!3v!evmr1qc!72v,vwutrc2v57r17v,ev3 qv;!e!r7fv3 qvcq,k!3uve3r7qv!evr7v3 qvwqkqe3!,kv q,zv57r17v,ev57r1 qcqfv!3v!ev1!3 v3,7qkqqcv3!;,7v!7v !evm,3 q3!wvwrkkqw3!r7nv2!b3qzv3rv !hvtuv3 qv,e2,cz!,7ev,b3qcv3 qvz,c5vqk;qevb,!kicqv!7v,wxi!c!72vbrcv3 qheqk;qefv3 qv3!hqve3r7qv!ev,kervr7vq,c3 nv1!3 v,v2crimvrbvercwqcqcenvwiccq73kuv2i,czqzvtuv3 q!cvkq,zqcnv3 qv,7w!q73vr7qfv3 qvk,e3ve3r7qnv3 qverikve3r7qnv ,evuq3v3rvtqvbri7znvti3v!v ,;qvmi3vhuvb,;ric!3qvz,i2 3qcnv2,hrc,nvr7v3rv!3ev3c,!knv,7zve qv ,ev7r3vz!e,mmr!73qzvhqvuq3fvr7wqv,ve3r7qv!evwrkkqw3qzv!v1!kkvtqv,tkqv3rv ,c7qeev!3evmr1qcvie!72v,v2,i73kq3v!v ,;qvrczqcqzv3 qvz1,c;qevr7v7!z,;qkk!cfv3 qv3!hqvbrcvwreh!wvrczqcv!ev7q,cnv,7zv1 q7v!vwrkkqw3v,kkv3 qve3r7qev!v1!kkv rkzvi7!;qceqv!7vhuv ,7zefv i2ev,7zv5!eeqenvv3 ,7re"

The above is the cryptic message we have intercepted. The characters are alphanumerical, from the English language and include some scattered punctuation. We also note common characters such as 7, 3 and v, but there is no hope in solving this by hand. Luckily, Graduate school taught us about MCMCs and we feel it might be a good fit for the problem. There are 7 steps to the MCMC algorithm, which are as followed:

Step 1 — Pick a random solution.

Step 2 — Apply the solution to the problem.

Step 3 — Evaluate how well that worked.

Step 4 — Make 1 adjustment to the solution.

Step 5 — Apply the new solution to the problem.

Step 6 — Re-score our solution.

Step 7 (a) — If the new solution has a better score, it becomes the current solution and we repeat from Step 4. Otherwise:

Step 7 (b) -Calculate the ratio between the new solution and the previous, then pick a random number between 0 and 1. If the ratio exceeds that number, accept the solution. If it does not, reject the solution. Either way, we repeat from Step 4.

Note, Step 7 (b) allows for a greater percentage of the potential solution space to be explored by sometimes allowing “worse” solutions that may eventually provide a better answer. The probability of accepting these “worse” solutions is dependent on how much worse they are. This is often referred to as the “local minima problem”.

However, there’s one problem: how do we measure how well the solution worked? We need an empirical score that can tell us which of our two solutions is more likely to be the real decrypted message.

To do this, we look at neighbouring letters. For instance, in the crypted message one of the first strings is “!kzcq7nv3”. We could calculate how likely in the English language it is for ‘k’ to follow ‘!’, or ‘z’ to follow ‘k’, and so on. Then if we were to randomly change a letter, we could see if the new letter pairings are more or less likely to be from English words. To get these pair probabilities, we will need a huge corpus of English words — preferably one that includes Marvel words such as Avengers, Thanos or Hulk. To do this, we can scrape the entire Marvel Wiki (see GitHub) and count how often each letter will follow another.

Python dictionary containing the frequency of letters that follow ‘b’ in the Marvel Wiki.

From this, we find the letter pairing ‘ba’ 41,943 times whilst ‘bn’ only appears 141 times (“subnet” & “Kly’bn”, if you were wondering). We can then convert these counts to percentages. As these percentages tend to be very small, it makes sense to take their logs as to somewhat normalise their values. This will turn a value such as 0.0001% into -4. Now we have a dictionary of every letter pairing and their respective log-likelihoods we can evaluate any attempt at decrypting the message by simply adding all the log-likelihoods of each letter pairing together. For instance, if we evaluated the string ‘hello’ the log-likelihood would be -8.6, whilst the string ‘z9kk3’ would have -32.6. This is due to the fact that in the Marvel Wiki (or any English text), ‘he’ is far more common than ‘z9’, and so on.

Given we now have a function to evaluate any potential solution, we can implement the MCMC method to attempt to decrypt the message, the code of which is above.

We start with Step 1 by randomly choosing a solution and evaluating how well this scores. Applying the first solution renders the following “decrypted” message:

“i;1vd5awx47;.vx;.d x;vnq;1c x;.c;x:x1!.x; i;8wxn.;h5n4;n4a;t!5td5; i;”

This has a log-likelihood of -11,396. We then make one change to the solution by switching a letter pairing, re-score the model and determine whether this has made the solution more or less likely. Rinse and repeat. By the 1000th iteration the score has improved dramatically to -4,792 and the decrypted message now reads:

“fyalpimoc nhasp asif aprtalef asea j lus afyagc rsavmrnarnoabumbimafya”.

It’s still far from readable, but the spaces look natural and the numbers/ improper punctuation have vanished.

By the 3,000th iteration the message starts to look almost English, with a score of -3,891:

“by gpildren, spe sibe pat gobe so exeguse by hreas clan and mulmil by”

On the 7,000th attempt, we can read the opening line of the message:

“my children. the time has come to execute my great plan and fulfil my”

It takes until the 24,000 iteration for the full stop above to be correctly switched with a comma, and the score to converge on its global optimal at -3,451. The final solution is as followed ( WARNING — partial spoilers for a few Marvel films!):

My children, the time has come to execute my great plan and fulfil my destiny. The plan, as you know, is to use the cosmic entities known as the infinity stones to wipe half of the universe out of existence so that the other half may thrive. Our greatest challenge has been finding the stones in this vast universe, but now they are within our grasp. The space stone, knows also as the Tesseract, resides on Asgard. after Loki's failure on earth, Thor has taken it and put it in Odin's vault. The power stone is on Morag. I have ordered Ronan the accuser to retrieve it for me, in return I shall destroy Xandar. The mind stone is on Earth, I put it in the staff I gave to Loki, but now I have heard that it is powering a cyborg known as the Vision. The reality stone is on the celestial head known as Knowhere. It is with Taneleer Tivan in his pathetic collection, gifted to him by the Asgardians after the dark elves failure in acquiring it for themselves. The time stone is also on earth, with a group of sorcerers, currently guarded by their leader, the Ancient One. The last stone, the soul stone, has yet to be found, but I have put my favourite daughter, Gamora, on to its trail, and she has not disappointed me yet. Once a stone is collected I will be able to harness its power using a gauntlet I have ordered the Dwarves to make for me on Nidavellir. The time for cosmic order is near, and when I collect all the stones I will hold Universe in my hands. Hugs and kisses, Thanos

(With thanks to the ultra Marvel geek Bob Gard, for writing this for me.)

You quickly print off the message and whiz it over to your boss. Within hours the Avengers have assembled and are drawing out plans to protect the stones and defeat Thanos. You are swiftly promoted to Senior Data Scientist and given a healthy raise.

But, as per the Marvel comics, there exists infinite universes with infinite possibilities. In all of these, were you successful in decrypting the message and defeating Thanos? To test this hypothesis, we can re-run the MCMC algorithm 100 times and see how many times it converges to the correct solution.

The log-likelihoods at each iteration of the algorithm for 100 independent repetitions.

As we can see from the above, it is not always successful at reaching the global maximum. In fact, we can look at the distributions of the final log-likelihood to see how many of these 100 times we were close to the optimal.

The frequency of the final log-likelihoods for the 100 independent repetitions.

From here, we find that in exactly 50% of the 100 Universes we would have correctly decrypted the message and saved Earth. But what about the other 50 times? What message would they have seen? The table below gives you an idea of what solution you would have presented to your boss, Nick Fury, depending on which score the algorithm converged on.

However, it is likely that with enough iterations most of these would have eventually found their way to the right answer.

The exception being the last one, which has fallen for a trap that is nearly impossible to recover from. The common letters have been almost all replaced by numbers. Given that numbers most likely follow other numbers, any attempt at replacing a single character with a letter would be met with a disastrous decrease in log-likelihood that would almost never be accepted.

Take the word “children”, for instance. In the lowest scoring attempt, it was decrypted as: “.46!?327,”. If the algorithm had switched the “2” for the correct character “r”, it would have not improved the score given “32” and “27” will have occurred more often than “3r” and “r7” in the Marvel wiki.

A simple solution for this is to repeat steps 1–3 (choosing completely new random solutions and evaluating them) a handful of times at the start of the algorithm, then continue with the highest performing of these. By including this, and by increasing the iterations to 100,000 we were able to converge close to the solution (1 or 2 incorrect characters) 100% of the time. This saves not only our Universe, but all the multi-verses that have found themselves in the same predicament.

GitHub: https://github.com/JackWillz/Projects/tree/master/MCMC%20-%20Enigma%20Thanos