Fixed and broke stuff.
added graceful shutdown
Fixed an issue that caused the crawler to max CPU
added cache manager
fleshed out cache manager
cleaned up utils
Made the CacheManager static.
Pushing to GH