cleaned up crawling manager

condensed thread handling