dxr: Add JSON serializer

It will be used in the JSON-based interface to DXR data.
Update sample config file

The "revision" key is used to customize revision ID extraction
across different kinds of repository
dxr: Update JsonOutput to write directly to a FD

No string accumulators are necessary as the data is usually
serializable right away.
dxr: start separation of queries into a different file

This is so both the html and json formatters can make use
of the same data.
www: Make search.cgi import the queries module

This also cleans up the code slightly
www: Add json.cgi for JSON output

This cgi takes the tree/string/query GET parameters. tree and
string are similar to query.cgi, being respectively the tree to
query (ie: mozilla-central) and string the search term(s), query
so far may be one of fts/regex/files to trigger the different
queries.
dxr: Add update.sh

This script is based on prior work from Christian Dywan.
It pulls, builds and indexes a source code repository as
specified in the dxr.config file.
setup-env: Use dxr.config and read parameters from there

Instead of "<srcdir> [<objdir>]" setup-env.sh now takes
"<dxr.config> <treename>" as parameters and then reads
the relevant parameters (srcdir, objdir, etcpp) from the
dxr.config file.
Based on a patch from groleo <groleo@gmail.com>.
Use absolute path for sqlite-tokenizer building & loading

Int setup-env.sh and dxr-index.py prefix the path to the sql
tokenizer library with DXRSRC.
This fixes the tests framework.
update README

The way to operate with setup-env.sh has changed
dxr-index: fix index.html generation to happen on the temporary dir

This is so all indexing data is generated on the temporary directory,
which then replaces atomically the target directory.
dxr: Do not parse the configuration file when building each html file

Instead reuse the tree configuration passed onto the constructor, as it
contains the keys we need.
www: Export more JSON queries

Queries of type [macros|functions|variables|warnings|callers] are now
handled.
Merge branch 'master' into lanedo-changes
Merge pull request #20 from lanedo/lanedo-changes

Lanedo changes
fts: use case-insensitive matching

This is for parity with MXR, so search results are similar
www: enable middle clicking on the code to start a new search

The left button still shows the info popup on the token clicked, whereas
middle clicking would let it open a new tab/window with the token search
results.
update.sh: Ensure the config file envvar isn't overwritten
dxr: Perform more operations directly against the DB

The indexing phase has been refurbished so more data hits directly
the database and less is retained in memory. The clang plugin has
changed too to adapt to this, resulting in overall lower times for
indexing (from ~3h30m to ~40m) and lower memory usage (peaking at
25% of the original one)

The database schema had to change too in order to allow for perfomant
queries as well for HTML generation, all *loc fields have been
replaced by file_id/line/col triplets, with a foreign key on the
files table, so searching for a given file doesn't result in a full
table scan. With these changes, dxr-index.py is able to generate an
average of 37 HTML files per second.
www: Update frontend to schema change

All queries upon *loc fields have been updated to the new schema.