Fix for HER-1962 NPE from missing sheet
* SheetOverlaysManager.java
    getOverlayMap(String) - return null if sheet missing instead of triggering npe
* KeyedProperties.java
    get(String) - check for null return value from getOverlayMap() and log warning
* CrawlURI.java
    markPrerequisite() - call setPrerequisite(true) on the prerequisite CrawlURI
* CrawlURI.java markPrerequisite() - call setPrerequisite(true) on the prerequisite CrawlURI
Merge branch 'master' of github.com:internetarchive/heritrix3
HER-1964 record whois server ip address in warc
* FetchWhois.java
    fetch() - record server ip address in CrawlURI attributes list
* CoreAttributeConstants.java
    new string constant A_WHOIS_SERVER_IP, key for whois server ip address CrawlURI attribute
* WARCWriterProcessor.java
    writeWhoisRecords() - include warc header WARC-IP-Address
HER-1963 Unvisited hosts being written twice into H3-generated hosts-report.txt
* HostsReport.java
    write() - remove obsolete code described by this no longer correct comment - "StatisticsTracker doesn't know of zero-completion hosts; so supplement report with those entries from host cache" - StatisticsTracker does know of zero-completion hosts
.classpath
    remove unusued, unneeded dependency asm-3.2.jar, see http://tech.groups.yahoo.com/group/archive-crawler/message/7384
[HER-1976] H3: query-portion ignored when checking against robots.txt rules (using only path)
* RobotsPolicy.java: use UURI.getPathQuery -- like H1 does
[HER-1981] H3: overlay setting of alternate queue-precedence (via
BaseQueuePrecedencePolicy.basePrecedence) doesn't stick
(CrawlURI) don't null overlayNames in processingCleanup
(WorkQueueFrontier) move queue precedence recalc to only in handleQueue,
not all (post-wake) reenqueues
[HER-1976] query-portion ignored when checking against robots.txt rules
(using only path)
(RobotsPolicyTest) added for case that failed pre-fix
(RobotsTxtTest) added case similar to problem (but was already OK at
RobotsTxt level)
(RobotsPolicy, *RobotsPolicy) rename method to reflect new behavior
Fix for HER-1983 mangled dns records in w/arcs
* FetchDNS.java
    recordDNS() - use local variable for dummy read buffer, instead of member variable that was shared among threads, since RecordingInputStream.read() used the passed buffer as a work area
add processErrorOutlinks setting to CandidatesProcessor
default false is old behavior, skip candidates-handling of outlinks from response codes <200 and >=400
if true, these outlinks will be treated the same as others (get scope-tested and enqueued)
HER-1984 save script state - implement by adding a map to the application context for arbitrary use, and make sure the app context is available in all scripting environments
* PathSharingContext.java
    new member variable ConcurrentHashMap data and accessor getData()
* ScriptedProcessor.java, ScriptedDecideRule.java
    make appCtx available to scripts; also remove unused member sharedMap
* ActionDirectory.java
    formatting fix
Merge branch 'master' of github.com:internetarchive/heritrix3
* profile-crawler-beans.cxml
    commented-out default value for new CandidatesProcessor settings processErrorOutlinks
Sort script engines alphabetically so that they appear consistently.
Note that the name of the engine may be different than the name of the language
it parses. A good example is the Rhino script engine that parses javascript.
Fix for HER-1985 H3: SurtPrefixDecideRule forgets learned/seed-derived/seed-directive SURT prefixes in checkpoint-resume
* SurtPrefixedDecideRule.java
    make class implement Checkpointable, save surt prefixes to json on checkpoint and load them on recover
* profile-crawler-beans.cxml
    make main SurtPrefixedDecideRule a top-level bean so that it can be checkpointed
* Checkpoint.java, CheckpointService.java
    add some FINE level logging during checkpointing and recovery
Merge branch 'master' of https://github.com/travisfw/heritrix3
Fix problems handling crawl job paths with characters that require url-encoding, since we sometimes refer to them with file: urls
* PathSharingContext.java, CrawlJob.java
    use java.net.URI to convert paths to and from file: urls
Merge branch 'master' of github.com:internetarchive/heritrix3