Track links in the lucene database, to reduce memory usage.
Various fixes and improvements.