andris9/mimelib — pull request #18 by asutherland (2012-11-30 16:06:52+00:00)
Title: Do not coalesce MIME encoded-words in order to avoid base64 breakage.

decodeMimeWords() currently merges adjacent MIME encoded-words with the same encoding type and character set. This does not reliably work for base64 encoded words. For example, a variation on the following was witnessed in the display name portion of a mail address in a message composed by Thunderbird (I changed the non-accented characters since the person's name is not actually relevant):
```
=?UTF-8?B?U3NzxZsgTGHFgmFsYQ==?= =?UTF-8?B?bGFsYQ==?=
```
ends up as the following alleged base64 string that gets fed to new Buffer(..., 'base64'):
```
U3NzxZsgTGHFgmFsYQ==bGFsYQ==
```
which results in the following test failure from the test case added in this patch (without the fix):
```
AssertionError: 'Sssś Lałalalala' == 'Sssś Lałala\u0006�\u0016�\u0010'
```

I belive the goal of the code was to accomplish the removal of whitespace between consecutive MIME encoded words, plus an attempt at optimization. This patch just transplants the logic from mailparser's _replaceMimeWords function for whitespace removal since it's much more terse.

One could argue that the optimizing logic should just not merge 'B' encoded words. However, I think given that RFC2047 says "The 'encoded-text' in an 'encoded-word' must be self-contained", one could argue that coalescing the encoded words will lead to different behaviour in cases where the data is not self-contained and so decoding the encoded words separately is more correct. I favor getting rid of the optimizing logic because it's more complex and has no comment justifying the coalescing logic.