DigitalPebble/storm-crawler — pull request #13 by jakekdodd (2014-09-23 16:21:47+00:00)
Title: added domain blacklist filtering

This pull request adds a domain blacklist URL filtering capability, ported from Apache Nutch 1.9. Included are:

1. A DomainBlacklistURLFilter class
2. Several domain-related utils that support the filter
3. New domain-related methods in URLUtil
4. A domain-suffixes.xml resource
5. An example domain blacklist resource