changes to crawler; however, we will have to abandon the dotabuff crawler due to dotabuff's robots.txt