HER-2056 new option startNewFilesOnCheckpoint on writers, true by default for backward compatibility
avoid NPE resuming from checkpoint if "stats" is {} in warcWriter checkpoint json
Include in checkpoint data a list of ".open" WARCs and their current raw on-disk length, so the exact extent of pre-checkpoint content can be deduced/reconstructed. Also when flushing or closing writers, block to wait on any that remain active. (In context of heritrix crawl this is not normally necessary because of the dispositionInProgressLock managed by the frontier, but seems like a good idea in case WriterPool is used elsewhere.)
removed ThreadLocalHttpConnectionManager -- it has been moved to ia-web-commons
Merge branch 'master' of github.com:internetarchive/heritrix3
throw exception when host cache returns 0.0.0.0 for any host (trying to track down a strange problem)
oops, avoid npe
Adam's responsive UI
remove unused engine.css
new CrawlURI annotation "duplicate:digest" to simplify handling dups in stats, etc
bump version to 3.2.0-SNAPSHOT since there are some incompatible api changes since 3.1.1 (requested by kris sigurðsson)
try to avoid nearly endless loop of "java.lang.IllegalStateException: Can't open a cursor Database was closed." when startup fails and dumpPendingToLog is attempted
don't let the exception from commit df2a313797b15d01e3959e309842b25ebb92ca91 derail the whole crawl job
moving org.archive.net.PublicSuffixes to ia-web-commons
moving a bunch of stuff from heritrix-commons to ia-web-commons so that wayback doesn't have to depend on heritrix-commons
test fix (in ia-web-commons) for HER-2059
fix report links
abbreviate maven repository clauses, reduce redundancy, and let default settings prevail, such as updatePolicy=daily, pointed out by Erik Hetzner
useProjectArtifact=false to avoid [WARNING] Cannot include project artifact: ... it doesn't have an associated file or directory
Merge branch 'master' into nlevitt-master