mozilla/pdf.js — pull request #4790 by fkaelberer (2014-05-13 15:21:57+00:00)
Title: [WIP] Differential cmap compression

When I compressed cmap files with winrar, I noticed that they compress 2x better when put in a *solid* archive, which means that the file contents are very similar.

This PR makes use of this fact by storing cmap files differentially, that is, a cmap file A is either stored normally, or it is stored as a patch that contains the necessary data to recover A from a similar file B.
This reduces the combined size of the cmaps to 608k (53% of 1150k) and the size of pdf.js.xpi from 2MB to 1.5MB.

The patches are computed using a third party diff tool ([cemerick/jsdifflib](https://github.com/cemerick/jsdifflib)) which is a partial port of python's difflib.

The dependencies are chosen optimally with respect to the output size, which takes over an hour to compute (because each pair of files is diff'ed), but because that has to be done only if the cmap files change, I didn't care much about performance optimization.

The reconstruction of the original files is very simple. Each file starts with a `filename` string. If `filename` is empty, the rest of the file is stored normally. Otherwise, it `filename` points to a basefile from which the cmap is to be reconstructed. Reconstrucion commands are either
- `copy`, followed by a `start` and `length` number, which means that the respective data has to be fetched from the base file, or
- `insert` followed by a `length` number and the corresponding number of bytes to insert.

(Since the `copy` and `insert` commands are always alternating, the commands themselves do not have to be stored.) Strings and numbers are stored and restored using the existing functions in [cmapscompress/compress.js](https://github.com/mozilla/pdf.js/blob/master/external/cmapscompress/compress.js) and [cmaps.js](https://github.com/mozilla/pdf.js/blob/master/src/core/cmap.js).

I marked this as work in progress, because I have still some questions / TODOs that I wanted to check before polishing this.
- [ ] Does the license of jsdifflib allow usage in pdf.js?
- [ ] Are there any drawbacks I didn't think of?
- [ ] Do we need more test for this? Can somebody provide them?
- [ ] Reduce number of dependencies to ~1-3 (currently, each file depends recursively on ~10 other files)
- [ ] update [cmapscompress/Readme.md](https://github.com/mozilla/pdf.js/blob/master/external/cmapscompress/README.md)
- [ ] cleanup code / commits