Added reference to Norconex Committer IDOL.
Brought back async writing to default URL database for increased
performance.
New TODOs.
New todo.
Upgraded Apache HttpClient to 4.3.x (and other libs) and did a fair bit
of refactoring to accomodate this.  Removed deprecated classes from 1.1.
Many more changes/additions (see changes.xml).
The empty sub-folders contained under the "download" folder are now
periodically deleted.  This speeds up directory scanning and increases
performance on large crawls.
Added NullAppender.
Now ensures that robots.txt agent matching gives priority to the most
specific match (as opposed to the first match).  Sitemaps detected in
robots.txt are also preserved for sitemap resolving.
HTTP header checksum and document checksum are now added to the document
metadata as HttpMetadata#CHECKSUM_HEADER and HttpMetadata#CHECKSUM_DOC.
Fixed sonar issues.
Dependency update.
updated dependencies.
Web-site upgrade, preparing for release.
Using memory-mapped files for MapDB.
Removed mmapFileEnableIfSupported() which is suspected to cause OOM
exception.
Preparing for release.
Preparing for release.
Merge branch 'master' of https://github.com/Norconex/collector-http.git