Refactored cluster crawler interface
 - Made the cluster crawler interface a bit more generic
 - Allowing use with the upcoming instance crawler.