Why Base64 does not always contain + and \
By: Daniel Nutting
Tags: Security Analysis, Infosec, Base64

Have you ever noticed, when analyzing Base64’d data, that human readable text almost never, ever produces Base64 with `+` or `\`? Oh, you have!? Well, keep reading.

It’s important to understand that each character in a Base64 string is really just a representation of six bits. For a standard B64 alphabet, `A` represents binary 000000, `B` represents 000001, `Z` represents 011001, `a` 011010 and so on.

`+` represents binary 111110 and `\` represents 111111. Notably, these are the only two characters within the character set that represent five consecutive 1s that start in the first position.

So, in other words if I have Base64 “WGRa” I can convert each B64 character to binary, and concatenate:

010110 | 000110 | 010001 | 011010

To make bytes, I keep the bits in the same order, and just change the grouping from 6 bits to 8 bits:

01011000 | 01100100 | 01011010

Then, if I want to represent those bytes in a human readable format, I look up each 8 bit group on an ASCII table. To simplify the example, we’ll focus on basic ASCII (not extended, not unicode), because most human readable text is based on that character set.

Comparing to an ASCII Table, the first byte represents the ASCII character `X`, then `d`, then `Z`. Here’s a quick link to a table: ascii-code.com.

Notice that some bit pairs will change which group they’re in. In this example all of the bits from B64 group 4 moved into ASCII group 3, and the first four bits in B64 group 3 move into ASCII group 2. The bits always move in pairs.

So, let’s go in the opposite direction now. Let’s convert ASCII “Hi Mom” Into Base64.

Hi : 01001000 01101001 00100000

Mom: 01001101 01101111 01101101

Now let’s change the grouping from 8 bits to 6 bits.

010010 000110 100100 100000

010011 010110 111101 101101

And we look up each of those 6bit groups to produce SGkgTW9t.

Now what about those `+` and `\` characters I originally brought up? Well remember, those require 5 or 6 consecutive 1s, starting in the first position in that 6 bit group. That means the source of those bits needs to come from ASCII that can reorganize appropriately.

The easiest example is when the last 6 bits of an ascii character are either 111110 or 111111. ASCII has three printable characters that fit this description `>`, `?`, and `~`.

What about ASCII characters starting with those bits? Well that’s extended ASCII, but it might include characters like, ø, ü, and my personal favorite þ.

Another approach to generate 5+ `1`s could be the last two bits from one 8-bit group, and the first 4 bits of the next 8-bit group. So something like:

01010011 11100000 splitting to 010100 111110.

Well, the issue is as soon as you start an 8 bit group with a 1, you are in the extended ASCII. The ASCII schema originally only covered 7 bit groups back in 1963, and RFC 20 decided to carry ASCII in 8 bit groups whose high order bit is _always_ 0.

This means if the source material only uses standard ASCII characters, only `>`, `?` and `~` can generate a + or \ in Base64.

More so, those characters need to be placed just right so that the 6 right most bits create their own group.

That means only every 3rd ASCII character could lead to those tricky `+`s and `\`s.

?AA —B64Encode→ P0FB

A?A —B64Encode→ QT9B

AA? —B64Encode→ QUE/ A~A>BB —B64Encode→ QX5BPkJC

AA~BB> —B64Encode→ QUF+QKI+

`+` and `\` characters are surprisingly rare in Base64 from human readable text, because those ASCII characters are rarely used, and rarely in the right position when they are used!

Conversely, strings of 5 or more 1s in a data blob ISN’T that rare. So, this gives you a decent reason to immediately decode a Base64 string if you don’t see either of those characters.