Improved robots.txt handling

This PR adds a new mechanism for robots.txt caching and filtering.

Among the changes are:

- A new thread-safe, in-memory singleton cache for robots rules
- A RobotsURLFilter
- Moves the getCacheKey() method to the cache implementation because it's used in several locations
- The ParserBolt ensures that the robots rules are loaded into the shared cache, which means conditionally refetching and parsing the domain's robots.txt file