separated hq from local hertirix3 repo, fixed up layout
H3 module for deduplicating with WBM index (with test case). Added EasyMock to project dependency.
updated queryURL to production WBM.
support mixing fetch histories from multiple sources (hbase, WBM).
HER-2039: Initial work. Eliminated Link class and dealt with resultant
errors. Not tested any further than that.
moved hbaes de-duplication code to heritrix-contrib.
chnaged groupId to address duplicate groupId warning.
WbmPersistLoadProcessor.java: fixed indentation.
WbmPersistLoadProcessor: make buildURL() more flexible. now uses
template string with placeholders. no hard-coded parameters.
updated WbmPersistLoadProcessor for cookie-based authentication.
allows turning gzip compression on/off.
main for quick test.
WbmPersistLoadProcessor: add simple performance metric. drop confusing
metric failedCount, add missedCount and errorCount.
Basic revisit notation framework
Simplified revisit writing. Uses revisit profile attached to CrawlURI.
Ensure CrawlURI discards RevisitProfile after it has finished, in case
the same CrawlURI is issued again.
IdenticalDigestDecideRule now consults the new revisit profile mechanism
on the CrawlURI to determine if it represents an identical digest
revisit.
Corrected javadoc
Profiles refactored, shared elements in AbstractProfile.
WARC-Refers-To-Date now set as long and correctly formated in
getWarcHeaders.
Revisit profile is now populated with original URI and date.
Store refersToDate in String format (as intended to write in WARC).
Retain long setter for convenience.
Revisit headers now come correctly formated from the revisit profile.
ContentDigestHistoryLoader now sets the revisit profile on the CrawlURI
after determining that it is a revisit.