fix automatic JFlex download

closes #15
allow to fetch contents of renamed files in Mercurial

closes #4
closes #13
Combine Hash2TokenStream and List2TokenStream
be more specific about required install of SCM environment
Make Iterable2TokenStream a Reader.

Iterable2TokenStream splits the input into tokens in a way similar to
PlainSymbolTokenizer, but not identical. If it is a Reader instead, the
input could be passed on to PlainSymbolTokenizer to produce tokens in
a consistent and uniform way.
Change the analyzers so that they don't keep the entire file in memory.

Instead of passing a stream to the analyze() methods, and have them
cache the entire contents in a byte or char array, open a new stream
each time the source file should be read.

Also, write the xref files from the analyze() methods. In many
analyzers, this avoids the need for building up a data structure that
holds the full xref output in memory.

This fixes #8.
fix duplicate documents and tests for add/remove files to index
this fixes #7
Fix srcdir for the jrcs build target.

Since the old srcdir was a parent of the actual source root, Ant got
confused and always rebuilt jrcs, also in incremental builds when
jrcs hadn't been changed. Now, incremental builds will only rebuild
it if something has changed in the jrcs code.
fixes for FilePath searches being broken (in QueryBuilder),
cleanups, path tokenizer fixes and tests
fixes issue #9
lucene 4.2.1 , document new query parser, allow path regexp if just one and enclosed in /
auto format code
update lucene 4.2.1 in mvn build and fix packages
Improve cleanup of resources opened by the analyze() methods.

Before issue #8, there would be exactly one FileInputStream per file
that was being analyzed, and it was pretty easy to ensure that this
stream was closed. Issue #8 changed this, but it didn't add logic to
ensure that the extra streams were closed if there was an error before
the document was added to the index.

This change attempts to improve the situation by making sure that
readers or token streams associated with the fields in the Lucene
document are closed if something goes wrong.
Improve log formatters

Fixes the following issues:

- SimpleDateFormat is not thread safe, so the shared instance should
  be protected by synchronization.

- Logged exceptions are hidden on low log levels, even if the handler
  is configured to show messages on that level.

- IndexDatabase re-implements printStackTrace() in order to work around
  exceptions not being logged on low log levels.
introducing uuencode(5) support: makes Xref files much smaller (no more
random huge search-path links), plus should significantly offload the
index (all the uu and base64 stuff is ignored)
enable uuencode support
(missing from previous commit)
add scala support (just for xrefs, defs and refs are not in there)
lucene 4.3.0
ignore paths, not just filenames; fixes issue #38
opensolaris.org cleanup; closes issue #5