update guess at log line length based on real world data
remove a couple of crufty lines of code
make AbstractFrontier.needReenqueuing() public so it can be called elsewhere
new module AMQPCrawlLogFeed, sends crawl log lines in tab-separated format to amqp; refactor so that amqp code is shared with AMQPPublishProcessor by way of new parent abstract class AMQPProducerProcessor
move HostsReport.fixup(String) to CrawlHost.fixUpName() so it can be called elsewhere
include host in feed
expose forAllPendingDo() as public so it can be used elsewhere
dumpPendingAtClose for AMQPCrawlLogFeed, dumps pending urls with 0 status to the amqp feed, similar to Frontier.dumpPendingAtClose for the crawl log
switch to json instead of tsv, and split size/length field into two fields, contentLength (not including protocol headers) and size (includes headers)
use JSONObject.NULL for null values so that they appear explicitly in the json
move AMQP producer code to a separate class so it can be used by classes that are not heritrix processors
new class DecideRuleSequenceWithAMQPFeed publishes scoping decisions to amqp
add field "extra info" to DecideRuleSequence log, which is a json object with entries for host, via, source and hop path
rename AMQPCrawlLogFeed fields to match James's names in druid (looks like there's no way to translate)
change DecideRuleSequence extraInfo field name 'source' to 'seed' (for reason external to heritrix)
Merge pull request #99 from nlevitt/aitfive-39

AMQPCrawlLogFeed, DecideRuleSequenceWithAMQPFeed, DecideRuleSequence.logExtraInfo