converted crawlQueue from array to hash, so we can check easily check for dups