Improved crawler (almost ready, needs testing).