jgranick/NME — pull request #13 by djcsdy (2012-08-04 02:00:05+00:00)
Title: Import MP3s into SWFs gaplessly wherever possible

Hello again :).

Apologies for the wall of text here and the fairly large amount of code I’m asking you look at and pull in. I hope that this description will cover everything relevant, but if there’s anything I can do to make your life easier please don’t hesitate to let me know.

The attached changeset contains a complete rewrite of the MP3 importing code for Flash. It includes the following improvements:

* Most importantly, wherever possible, it imports MP3 files without the characteristic silent “gap” at the beginning and end of the audio. Presently “wherever possible” means for MP3s encoded with LAME, read more on this below.
* It checks the header of every frame to ensure that all frames are suitable for embedding in a SWF, not just the first frame.
* It checks that the format does not change between frames in various ways that are arbitrarily disallowed by Flash Player.
* It calculates the length of the MP3 correctly in an admittedly-unlikely case where the old code would fail (variable MPEG version between frames — which is permitted, apparently).
* It discards any data that would be ignored by Flash, such as ID3 tags, invalid frames and other garbage non-MP3 data (it would be possible to warn when encountering this stuff, but that would be useless in practise since almost all MP3 files include some non-MP3 data).

As you may be aware, MP3s imported into SWFs by NME (or Flex, or most other similar tools) suffer a characteristic silent or near-silent gap at the beginning and end of the audio. This gap is on the order of 100 or 200ms in both cases, but the exact length varies depending on various factors. The reason for this is a technical limitation with the MP3 format, which means that, using the MP3 standard alone, it is in practise impossible to encode audio without a short gap at either end.

My Github project [nme-gapless-mp3-example][2] demonstrates the problem. It plays a short MP3 on loop. Clone and build the project without my patch, and there is a significant silent gap between each loop. Build with my patch and the gap is gone and the loops are seamless.

* [Prebuilt sample with gaps][3]
* [Prebuilt sample without gaps][4] (built with this patch)

These gaps are annoying when playing sound effects, since there is a significant perceptible delay between an action and its corresponding sound-effect (unless you use some inexact method to compensate for the delay in code). It’s even worse when playing back a loop, since there is a significant gap between the end of one loop and the beginning of the next.

Fortunately, most MP3 encoders use some non-standard but documented method to embed the information we need to chop off the correct number of samples at either end of the decoded audio data. LAME and compatible encoders embed this in a so-called ‘Info’ tag, and Fraunhofer encoders embed it in an ‘Original File Length’ field.

Also fortunately, Flash provides a simple way to embed equivalent information into a SWF alongside embedded MP3 audio data. Therefore, all we need to do is to translate the gapless playback information from the input MP3 and embed equivalent information in the SWF. This is an official, documented, but often overlooked feature of the SWF format. It is also the ‘secret’ technique that the Flash authoring tool uses to embed MP3s without gaps.

These changes provide support for the LAME/‘Info’ method of embedding gapless playback information. MP3s encoded by LAME or compatible encoders — in practise MP3s produced by almost any open source or free tool — will be embedded without gaps.

MP3s encoded by other encoders are unaffected. They will still import just fine as before, but they won’t be gapless.

It would also be possible to add support for gapless information embedded by Fraunhofer encoders. Between Fraunhofer and LAME, that would cover practically every MP3 in existence. I plan to do so in the future but I don’t have any personal need for it, so it won’t be terribly soon.

The classes in `mpeg.audio` are copied from my project [mpeg.audio.hx][1]. That project is thoroughly tested, but it isn’t quite ready for general public consumption, so I haven’t released it on haxelib yet. It should however be more than adequate for the use it is put to here. Therefore, I’ve followed the existing practise with code from the Haxe formats library and copied the code into the NME source tree.

[1]: http://github.com/djcsdy/mpeg.audio.hx
[2]: http://github.com/djcsdy/nme-gapless-mp3-example
[3]: http://www.danielcassidy.me.uk/misc/nme-mp3-gaps.swf
[4]: http://www.danielcassidy.me.uk/misc/nme-mp3-no-gaps.swf