whitespace
Catch tracebacks thrown by workers, and report them in context, not just relative to where the error was collected in concurrent.futures. Refs bug 924631.

Stop reporting incorrect file IDs of the workers that failed.
Upgrade concurrent.futures to 2.1.4.

It doesn't seem to do any harm, and there have been some bug fixes since the last version.
Fix a NameError in dxr-build.
Add a "format" file, which says what version of instance the code requires.

This will let us (continuously) deploy new versions of the code that change the SQL schema, etc. For now, this just lets us rewrite the build script to depend on this file being here. Next, we'll make deploy.py exit if a suitably versioned instance isn't yet built.
style
Recognize includes in the compiler plugin. Rip out custom C++ tokenizer, now unused.

Don't guess them after the fact from Python.

This adds an `includes` table to the schema.

Also, tighten up some bare excepts to not catch control-Cs.
Enable git fsck.

Our new stage and prod boxes have new enough gits to support the --no-dangling option.
Wait to deploy new web app code until a new enough instance to support it has been built.
Add a diagnostic message to build.py.

Sometimes, the HTML building hangs. This time, it was before any workers completed. This will tell me whether it's hanging while submitting the jobs or whether, instead, as_completed() isn't working.
Upgrade to Pygments 1.6 so we don't have an infinite loop in the lexer while highlighting mozilla-central/gfx/harfbuzz/src/hb-ot-tag.cc.
Line-buffer the log files so we can reliably see what the last file processed by a frozen HTML worker was.
Retire our custom C++ tokenizer in favor of slurping up includes within the clang compiler plugin. Fixes bug 924631.

Many other improvements as well:
* Better diagnostic messages for failing HTML workers
* Deployment improvements, including schema version awareness and git hash verification.
* Use Pygments for C++ syntax highlighting. (#156)
* Upgrade concurrent.futures to 2.1.4.